Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Continuous Context Driven Test Improvement
Similar to Continuous Context Driven Test Improvement (20)
More from TechWell
More from TechWell (20)
Recently uploaded
Recently uploaded (20)
Continuous Context Driven Test Improvement
- 1. W12 Special Topics 5/10/17 13:45 Continuous Context Driven Test Improvement Presented by: Jeroen Mengerink Polteq Testing Services B.V. Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ info@techwell.com -‐ http://www.starwest.techwell.com/
- 2. Jeroen Mengerink Test consultant for the Netherlands-‐based Polteq Test Services B.V. Jeroen Mengerink has performed multiple TPI assessments worldwide. His technical skills allow him to team with developers in testing websites, APIs, and web services. Jeroen performs both functional testing and performance testing. In addition to his work for clients, he is involved with test innovations in agile. Jeroen teaches the Certified Agile Tester course and courses on agile, SOA, and cloud; coauthored Testing Cloud Services; and blogs at jmengerink.wordpress.com. Follow him on Twitter @AngusVB.
- 3. Jeroen Mengerink Continuous Context Driven Test Improvement
- 4. Continuous Context Test Improvement for any Situation Improvement of Improvement The Evolution
- 5. Continuous Context Driven Test Improvement Improvement of Improvement The Evolution
- 6. Then Today ßStructure Pioneering Structuring Op/mizing Unstructured Undisciplined Flexible Structured Disciplined Inflexible The Evolution of Testing
- 10. Prototyping Waterfall Agile/Scrum V-model Spiral model DEVOPS Iterative RAD TDD ATDD BDD W-model XP Continuous Integration/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB Packages SOA Devices Social media Virtualization Cloud Mobile Internet of Things Web Localization Big Data The Evolution of IT and Testing Then Today
- 11. Creative Business knowledge Management Agile Writing Tooling Cooperative English Team player Security SOA SAP Social Mobile Questioning Integration Scrum Api’s CD Process Improvement Techniques TMap DevOps Creative Tolerant CMMI ISO/IEC 29119 Android Sigma Usability Honesty Standards Adaptive RQMS engineering Outsourcing Coding SaaS BYOD Waterfall Leadership Telecom Embedded Billing Logistics Reporting Asser/ve Ar/s/c TEST AUTOMATION CI CD SBTM Virtualization Technical Diplomatic Curious IT-general Networks Infrastructure Critical PRINCE2 IREB
- 12. Distributed Business/RQMS Development Operations Technical Automation Mobile & Cloud “Things” e.g. Security Coding Automation Development Social Team player Communication Craftsmanship Testing Test management CHANGED
- 15. SPI and TPI Initiatives IEEE MIL-STD CMM PSP TSP TPI TI4Agile CMMI TMM TMMI Juran SEI BS 5750 TSM STEP V2M2 TI4Auto CTPI TPI-Next TQC Deming IDEAL ISO9000-9126 TIM TOM STEP CTP Cloutest
- 16. Models for Test Improvement • TOM • V2M2 • SQR • CTP • TI4Agile • TPI (Next) • STEP • TSM • TIM • TI4Auto • TMMi • CTPI • TPS • CMMI And more ……
- 17. Models for Test Improvement • TOM • V2M2 • SQR • CTP • TI4Agile • TPI (Next) • STEP • TSM • TIM • TI4Auto • TMMi • CTPI • TPS • CMMI And more ……
- 20. Are our TPI-Models still adequate? Prototyping Waterfall Agile/Scrum V-model Spiral model DEVOPS Iterative RAD TDD ATDD BDD W-model XP Continuous Integration/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB Packages SOA Devices Social media Virtualization Cloud Mobile Internet of Things Web Localization Big Data Then Today
- 21. And: • High scores – … but does it really help? • Great improvement plans – … but poor results • Solid improvement policy – … competing with BAU – … low acceptance (people) • Ivory tower – … resistance Improvement Center
- 22. Worse than the ivory tower!
- 23. More suitable improvement approach required • Situational – Context Driven • Continuous • Joint effort • Change proof – Proactive • …… Fast Flexible Lean Effective People DrivenPracticalSustainable
- 24. Continuous Context Driven Test Improvement Improvement of Improvement Required:
- 25. What helps? • Learning from the past • Using successful “recent” innovations – Agile – Scrum – Kanban – Exploratory – Context Driven – Collaborating – Continuous
- 26. Test Improvement at two levels Improvement backlog Assessment tasks Improvement sprints Improvement tasks Con/nuous Improvement Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Architecture Implementation
- 27. Test Improvement using Scrum Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Improvement sprints Improvement backlog Improvement Stories Architecture Implementation
- 28. Test Improvement using Scrum Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Improvement Backlog Sprint Backlog 2 - x weeks Sprint DoD Improvement increment Scrum DoD Improvement Stories Architecture Implementation
- 29. Why Agile/Scrum? • Small increments • Ease of Control • Ease of Change • Ownership of participants • Focus on people • Joint effort • Increased use of Agile • …… Is Agile/Scrum a must? • Don’t forget the implementation!
- 30. Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Improvement backlog Assessment tasks Improvement sprints Improvement tasks Con/nuous Improvement Roles Improvement Architect Improvement Owner Scrum Master Team members
- 31. Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Improvement backlog Assessment tasks Improvement sprints Improvement tasks Con/nuous Improvement
- 32. Architect level Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’
- 33. Test Improvement Intake Objectives – Time – Costs – Quality (product) – Quality (testing) – ………… improve “Agile Testing” – ………… improve level of Test Automation – ………… prepare for Outsourcing – ………… Or for commercial reasons – Score as high (mature) as possible Test Improvement Intake (Objec/ve – Scope – Approach matching)
- 34. Test Improvement Intake • Objectives – T, C, Q(P), Q(T), … – Commercial • Scope – Area of consideration • Organization, department, project, product, … • Test level • Test type (e.g. security, performance) • Agile • Testing Cloud services • …………… Test Improvement Intake (Objec/ve – Scope – Approach matching)
- 35. Test Improvement Intake • Objectives – T, C, Q(P), Q(T), … – Commercial • Scope – Area of consideration Test Improvement Intake (Objec/ve – Scope – Approach matching) Mark on the Horizon
- 36. Test Improvement Intake • Objectives – T, C, Q(P), Q(T), … – Commercial • Scope – Area of consideration • Approach matching – Context – Models – Selection Test Improvement Intake (Objec/ve – Scope – Approach matching)
- 37. Prototyping Waterfall Agile/Scrum V-model Spiral model DEVOPS Iterative RAD TDD ATDD BDD W-model XP Continuous Integration/Delivery MBT Exploratory CDT RST RSTM TMap ISEB ISTQB TMap-Next Packages SOA Devices Social media Virtualization Cloud Mobile Internet of Things Web Localization Big Data Context
- 38. Models – TPI Next – TMMI – STEP – CTP – GQM – …… – TI4Agile – TI4Automation – CTPI – SFAI-web – Belbin – …… Bound Tailor-‐made
- 39. Models Unbound
- 40. Models Unbound – Experience based – Heuristic • Questioning • Discussing – Implicit knowledge – Exploring
- 41. Using heuristics, etc. How do you feel? Ever felt this before? How about your work? Doing sports? How is your family? In the morning? When was that? Need to walk? At what level? Simular complaints? Before breakfast? … … … …
- 42. Heuristics for Test Improvement • Observation • Image of testing • History, earlier improvement attempts • Discuss potential solutions • ………
- 43. Models Unbound – Experience based – Heuristic • Questioning • Discussing – Implicit knowledge – Exploring – Idea raising sessions – Out-of-the-Box
- 44. Idea raising session • Organize a meeting (lean, informal, 3-10 people, < 2 hours, coffee, post-it’s) • Introduction – Define objectives – Set the right focus • Gather ideas – Individual “brain dump” – Use post-it’s – Explain, explore, question, discuss – Cluster and group ideas, … – Use Mind-mapping • Prioritize according to objectives – E.g. term, costs, impact • List results, actions, epics, stories, …… – Who, what, why, …
- 45. Advantages – Efficient – 100% focus – Invented here! – Morale – Buy-in
- 46. Models Unbound – Experience based – Heuristic • Questioning • Discussing – Implicit knowledge – Exploring – Idea raising sessions – Out-of-the-Box – Mind mapping – Good Practices
- 47. Models – TPI Next – TMMI – STEP – CTP – GQM – …… – TI4Agile – TI4Automation – CTPI – SFAI-web – Belbin – …… – Experience based – Heuristic • Questioning • Discussing – Implicit knowledge – Exploring – Idea raising sessions – Out-of-the-Box – Mind mapping – Good Practices Bound Unbound Tailor-‐made
- 48. Selection of Approach/Model • Objectives • Scope • Context • Resources – Budget – Time – TI/domain knowledge • Maturity (best guess) • Culture • Fast • Flexible • Lean • Effective • Situational • Continuous • Joint effort • Change proof • …… Test Improvement Intake (Objec/ve – Scope – Approach matching)
- 49. Architect level Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’
- 50. Assessment • Preparation • Collect information • Analysis • Define improvements • Plan and Report
- 51. Bound (Tailor-made)? • Apply Scrum for implementation • Translate recommendations into improvement stories Assess according to the model,but … Improvement backlog Assessment
- 52. Unbound? – Questioning – Heuristics – Feedback and react – Exploring – Idea raising sessions – Walk around – Test along – Attend meetings • Apply Scrum for implementation • Translate recommendations into improvement stories Typical Assessment Improvement backlog
- 53. Combine whatever contributes! – TPI Next – TMMI – STEP – CTP – GQM – …… – TI4Agile – TI4Automation – CTPI – SFAI-web – Belbin – …… – Experience based – Heuristic • Questioning • Discussing – Implicit knowledge – Exploring – Idea raising sessions – Out-of-the-Box – Mind mapping – Good Practices Bound Unbound Tailor-‐made Assessment Hybrid Improvement backlog
- 54. Architect level Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’
- 55. Improvement Backlog • Translate Improvement Suggestions into Improvement Stories (or Epics) As senior IT-director, I want to increase test efficiency, so that the testing cost is reduced by 20% As scrum team member, I want to automate the regression tests, so that the effort for regression testing is reduced As product manager, I want to increase the release frequency, so that we will be more competitive Con/nuous Improvement ‘Release’
- 56. Improvement Release planning • Divide and prioritize in logical portions for Improvement Backlog – Contribution value (objectives) – Ease of implementation – Dependency Con/nuous Improvement ‘Release’
- 57. Towards Implementation level Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Improvement Backlog Sprint Backlog 2 - x weeks Sprint DoD Improvement increment Scrum DoD Improvement Stories
- 58. Continuous Implementation • Refinement and prioritizing Improvement backlog Assessment tasks Improvement sprints Improvement tasks Con/nuous Improvement Improvement Backlog Sprint Backlog 2- x weeks Sprint DoD Improvement increment Scrum DoD
- 59. Continuous Implementation • Refinement and prioritizing • Towards tasks, effort and time Improvement backlog Assessment tasks Improvement sprints Improvement tasks Con/nuous Improvement Assess the level of unit testing Implement a testing dashboard Hire a test automation specialist Investigate open source tools for regression testing
- 60. Continuous Implementation • Refinement and prioritizing • Towards tasks, effort and time • Integrated with Business as Usual (BAU) • Competition with BAU • Improvement backlog Assessment tasks Improvement sprints Improvement tasks Con/nuous Improvement
- 61. Continuous Context Driven Test Improvement
- 62. Conclusion • Improvement of improvement is required • The situation sets the (hybrid) improvement approach • Continuous improvement integrates with BAU using Scrum • Tearing down the ivory tower Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Improvement backlog Assessment tasks Improvement sprints Improvement tasks Con/nuous Improvement
- 65. Test Process Improvement Optimizing the quality, costs and lead time of the test process • Quality: – Timeliness – Coverage – Insight – Control • Costs and lead time: – Risk-based – Early detection – Reuse – Collaboration – Automation Efficiency Effectivity Continuously
- 66. Objective Test Process Improvement? Optimizing the quality, costs and lead time of the test process Scoring as high (mature) as possible to show senior management or potential customers or
- 67. Objective Test Process Improvement Optimizing the quality, costs and lead time of the test process • Quality: – Timeliness – Coverage – Insight – Control • Costs and lead time: – Risk-based – Early detection – Reuse – Collaboration – Automation Efficiency Effectivity Continuously Scoring as high (mature) as possible to show senior management or potential customers
- 69. Acting Establishing Learning Using IDEAL Improvement Backlog Sprint Backlog 2- X weeks Sprint DoD Improvement increment Scrum DoD Test Improvement Intake (Objec/ve – Scope – Approach matching) Assessment Con/nuous Improvement ‘Release’ Initiating Diagnosing