Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QA Fest 2019. Михаил Чуб. Test Automation Сonsultancy

34 views

Published on

Why would you like to become a "Consultant". Where does it begin? And what to do if you're selected to consult a big Bank abroad :)
Successful Test Automation and Performance Testing Consultancy case study will demonstrate the whole process from finding the client through assessment, evaluation, and implementation to a happy client accepting the project.

Published in: Education
  • Leverage Test Automation Services to build automated test scripts using open source tools and test your website under peak load conditions. Maximize software quality. Get a free Proof of Concept. http://bit.ly/2lTjjzX
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

QA Fest 2019. Михаил Чуб. Test Automation Сonsultancy

  1. 1. Тема доклада Тема доклада Тема доклада KYIV 2019 Mikhail Chub TEST AUTOMATION CONSULTANCY QA CONFERENCE #1 IN UKRAINE
  2. 2. 2 HELLO Head of QA Competence, Infopulse Ukraine Lecturer, KPI MIKHAIL CHUB 16 years in QA 5 years in Education ISTQB ATM Hobby mikhailchub mikhail.chubqamania
  3. 3. NEW OPPORTUNITIES Stronger Boss – better loot BOOST EXPERTISE Work harder – get stronger MOM WILL BE PROUD Attaboy! APPLY ALL KNOWLEDGE Right now! WHY CONSULTING
  4. 4. CONTENTS 4 START ABOUT PROJECT & CLIENT Few words about project specifics, client and the scope.
  5. 5. CONTENTS 5 “WARM” PHASE CLOSURE • Acceptance • Maintenance “HOT” PHASE OPERATION • Assessment • Tools Evaluation • Proof of Concept • Training • Pilot • Strategy “COLD” PHASE INITIATION • Opportunity • Waiting • Hiring
  6. 6. CONTENTS 6 END AFTER THE PROJECT WHAT’s NEXT • New projects • Build competence • Adjusted templates LESSONS LEARNED SUMMARY • What was good • What was NOT so good
  7. 7. ABOUT CLIENT & PROJECT CONSULTANCY
  8. 8. ABOUT CLIENT 8 . Bank under transformation
  9. 9. ABOUT PROJECT 9 . Bank under transformation
  10. 10. ABOUT PROJECT 10 How to make it right AUTOMATED TESTING Teach how to do it right TRAINING How to make it right PERFORMANCE TESTING Implement few tests together with the local team so they are able to copy-paste progress further PILOT Evaluate and select the best suitable tool PROPER TOOLS Define strategy for auto and perf tests development SYSTEMATIC APPROACH
  11. 11. INITIATION CONSULTANCY
  12. 12. OPPORTUNITY 12 FIND RFP Make sure your Sales understand what to look for and what to propose UNDERSTAND DECOMPOSITION Ask questions, recall your past projects, identify high-level activities ESTIMATE PROJECT PLAN Estimate activities using your expertise and statistical methods PROVE CASE STUDY Collect proofs of your expertise, compile them to a convincing slides WIN PRESENT Present your approach personally to the potential client, be fair and convincing
  13. 13. ESTIMATION 13 . . .
  14. 14. PROJECT PLAN 14 JAN FEB MAR APR MAY JUN JUL ASSESSMENT1 TOOLS EVALUATION, POC2 AUTOTESTING TRAINING3 AUTOTESTING PILOT4 PERFTESTING TRAINING5 PERFTESTING PILOT6 MAINTENANCE7
  15. 15. OPPORTUNITY 15 FIND RFP Make sure your Sales understand what to look for and what to propose UNDERSTAND DECOMPOSITION Ask questions, recall your past projects, identify high-level activities ESTIMATE PROJECT PLAN Estimate activities using your expertise and statistical methods PROVE CASE STUDY Collect proofs of your expertise, compile them to a convincing slides WIN PRESENT Present your approach personally to the potential client, be fair and convincing
  16. 16. WAITING 16
  17. 17. HIRING 17 Explain complicated things in simple words ADVISORY SKILLS Able to establish plans, milestones, KPIs and follow them MANAGEMENT SKILLS Making people happier rather than writing good autotests COMMUNICATION SKILLS TECHNICAL SKILLS Solid hands-on experience
  18. 18. INTERVIEWING 18
  19. 19. INTERVIEWING 19
  20. 20. TEAM MEMBERS 20 AutoTesting Consultant Does magic with automated tests. Principal Consultant Recruits team of superheroes and makes sure that other superheroes remember about the Great Goal. PerfTesting Consultant Uses science to hurt.
  21. 21. OPERATION CONSULTANCY
  22. 22. ASSESSMENT 22 COMMUNICATION
  23. 23. ASSESSMENT 23 0201 03 400-500 ACTIVE USERS Core Banking INTEGRATIONS Open Account, Top-up Mobile MAIN USE CASE Give me all your money PURPOSE WinForms TECHNOLOGY Each 3 months RELEASE FREQUENCY STRUCTURE
  24. 24. ASSESSMENT 24 AUTOMATED TESTS Number of scripts implemented MILESTONES ARE MET PROJECT PHASES FINISHED IN TIME Priorities defined, Tools selected, Scope agreed TOOLS EVALUATION MATRIX Detailed tools comparison SCRIPTS IN GIT Developed and stored KPI METHODOLOGY Written and agreed
  25. 25. TOOLS EVALUATION 25 TEST COMPLETE RANOREX WIN APP DRIVER COLUMN ONE Technology COMPARISON CATEGORY CI IDE, Debug Maintainability Execution, Reporting Support TOTAL AUTOTESTING
  26. 26. TOOLS EVALUATION 26 LOAD RUNNER LOAD UI WAPT LOAD IMPACT Technology CATEGORY CI IDE, Debug Maintainability Execution, Reporting Support TOTAL NEOLOADJMETER PERFTESTING
  27. 27. TRAINING PRACTICE Gradually rolling to Pilot implementation THEORY For PerfTesting mostly TRAINING PROGRAM Intensive: 8d – Auto, 6d - Perf SELF-STUDY MATERIALS Video, Books, Forums, Chats, Articles, Other links
  28. 28. PILOT 28 AUTOTESTING + =>
  29. 29. PILOT 29 PERFTESTING
  30. 30. PILOT 30 PERFTESTING SATURATION POINT in Client Bank
  31. 31. CLOSURE CONSULTANCY
  32. 32. ACCEPTANCE • According to defined earlier: – Milestones – Deliverables – KPIs • Accept what is met • Agree on what is NOT met 32
  33. 33. ACCEPTANCE Phase Artifacts Milestones KPIs 1. Assessment 1.1. Communication Matrix + 1.1. Intro Workshop + 1. Milestones are met + 1.2. Project Priorities + 1.2. Meetings + 1.3. Priorities + 2. Evaluation 2.1. Tools Evaluation Matrix + 2.1. AutoTool selected, agreed + 2.1. >= 2 Test Scenarios are successfully automated for chosen system + 2.2. Costs comparison + 2.2. PerfTool selected, agreed + 2.2. >= 2 PerfTest Scenarios are designed for chosen system +- 2.3. Time-to-market comparison +- 2.3. Project scope agreed + 3. Methodology 3.1. Methodology document + 3.1. Methodology drafted + 3. Milestones are met +- 3.2. Training Program (Auto) + 3.2. Methodology finalized + 3.3. Training Program (Perf) + 3.3. Methodology agreed X 4. Autotesting Training 4.1. Projects from students in VCS + 4.1. Customer: classroom prepared + 4. Automation exam marks >= 65% X 4.2. Autotesting training projects completed by students + 5. Autotesting PoC 5.1. Implemented PoC project in VCS + 5.1. Autotests Architecture designed + 5.1. Implemented 5-10 P1 autotests for chosen system + 5.2. Test Execution Reports + 5.2. Test Data generator developed +- 5.2. Able to run autotests from CI + 5.3. Autotests Adopted for CI + 5.3. Atomation Test Reports are understandable + 5.4. Autotests Adopted for Reporting + 6. Perftesting Training 6.1. Training presentation + 6.1. Perftest Homeworks done + 6. Perftest exam marks >= 65% + 6.2. Team Profiling doc +- 6.2. Perftest theory exam conducted + 6.3. Perftest training final marks are set + 7. Perftesting PoC 7.1. Test Scripts + 7.1. Perftest activities on practice + 7.1. 5-10 Perftests are implemented for chosen system + 7.2. Test Data + 7.2. List of future PerfTest candidates prepared + 7.2. Able to run perftests from CI + 7.3. Test Reports + 7.3. Test Reports are understandable + 7.4. Lessons Learnt + 33
  34. 34. MAINTENANCE • Weekly calls • to solve problems 34
  35. 35. WHAT’s NEXT CONSULTANCY
  36. 36. BASELINE 36 Plum gummi bears. Apple pie yummy cake with candy SERVICE QUALITY Sugar gummi bears. Apple pie yummy cake with candy TRANSPARENT RELATIONSHIP Sugar plum gummi bears. pie yummy cake with candy TEST PROCESS REVIEW
  37. 37. ADDITIONAL OPPORTUNITIES AND SERVICES 37 BUILD COMPETENCE CENTER Help to establish competence MANUAL TESTING TRAININGS Help to increase competence on the Basic and Advanced level THREE NEW OPPORTUNITIES • Solutions Development • DevOps Consultancy • Additional Testing Services TEACH TEACHERS Implement methodology for further competence growth
  38. 38. SUMMARY CONSULTANCY
  39. 39. PROBLEMS 39 PROBLEMS AND THEIR RESOLUTION CONSULTANCY • Shared • Hardware isn’t close to PROD ENVIRONMENT • Only through terminal • No VPN access ACCESS • Hard to allocate internally • Hard to find externally HIRING
  40. 40. PROBLEMS 40 PROBLEMS AND THEIR RESOLUTION CONSULTANCY DOUBTSAUTHORITY • Local team • UI localization • Test Cases LANGUAGE
  41. 41. LESSONS LEARNED 41 TEMPLATES Reuse saves time PROJECT PLAN Is a must RESPECT – IS EVERYTHING Уважение клиента ЛУЧШЕЕ – ВРАГ ХОРОШЕГО Иногда можно быть проще БОЛЕЗНЬ А НЕ СИМПТОМЫ До автотестов еще нужно дойти CLASSIC TRAINING WITH THEORY Теория + Практика лучше чем одна Практика
  42. 42. THANK YOU QUESTIONS?

×