Online QADay 2022 #2 РОМАН МЕЛЬНИК «Testing in CD_ how we implement it, risks etc.» telegram: wwww.t.me/goqameetup fb: www.fb.com/goqaevent fb: www.fb.com/qaday.org linkedin: https://www.linkedin.com/company/goqa/ Сайт: www.online.qaday.org

1. Testing in CD: how we
implemented it, risks etc.
Roman Melnyk

2. About me
• QA Lead in Matic Insurance
• Former lecturer at Logos and OkteWeb University
• Ironman 70.3 finisher and marathon runner

3. Traditional testing
● Time-boxed
● Hand off centric testing (product handed off between engineers
during Development and QA phases)
● Focused on manual testing
● Quality process prevails over schedule and can impede the delivery
pipeline
● Poorly integrated into Continuous Delivery/Deployment process

4. QA challenges
● Inconsistency between DevOps
transformation and QA process
● Not clear testing strategy within
development sprints
● Limited focus on business
requirements (what vs how)
● Limited focus on the automation

5. ● Undisrupted testing on a continuous
basis focused on achieving
continuous quality & improvement
● Emphasises business expectations to
mitigate business risks
● Enables shift-left (test earlier) and
shift-right testing (test in production)
● Can’t be implemented without
automation on different levels (unit,
integration, system)
Continuous testing

6. Continuous testing

7. Continuous testing

8. Process highlights
● No dedicated testing of individual user stories (by QAs)
● Dev testing is added to development process
● Manual QA focus - E2E/Integration workflows, business scenarios
● Automation to become the mindset rather than tool
● QAs to participate in the Requirements management activities
Testing should be a collaboration between Dev, QA and Ops

9. ● Focus
● User stories
● Features
● Artifacts
● Test Cases
● Checklists
● Execution reports
● Approach
● Reactive
● Script based mostly
● Limited development testing
● Focus
● End-user workflows
● Integration workflows
● Artifacts
● Use-cases
● Risk/coverage reports
● Approach
● Proactive
● Exploratory-type (session-based)
● Comprehensive dev. testing
Traditional Continuous

10. First steps
• Pilot run with single team
• Peer review
• Sessions with PdM’s
• Automation sessions
• Developer testing
• Testing notes
• Datadog monitoring (non-
functional testing)

11. Risks & Challenges
● Enhanced cultural shift within the engineering teams
● Possible integration issues because of long code integration
● Limited ability to test specific feature branch
● Necessity to rigidly follow defined quality gates
● Increased quality risks during roll-out phase
● Possible increase of the implementation estimates

12. What we have now
● Average bug life from open to closed - 4 days
● Average bug fix - 1,5-2 days
● Created/fixed defects - 75/64
● Average releases per day - 10.6
● Average releases per month - 224
● Average lifecycle of a pull request - 1h