Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
There and back again
Our journey with QA Reports and metrics
1
Zbyszek Moćkun
Intro
2
Agenda
3
01 The Beginning
02 An unexpected journey / Define QA Reports and metrics
03 Find your Black Arrow / Automate QA ...
The beginning
Long long time ago
4
Bigger, bigger and bigger
• New clients from pharmacy, insurance and banking
• Which required more documentation and more ...
How to control several simultanuously projects?
• Governance on company level
• Deliver always high / expected quality
• S...
An unexpected journey
Define QA Reports and metrics
7
How to identify metrics?
• Metrics should encourage to ask questions not give the answers
• Measure the processes not peop...
9
1 x full width image & text
placeholder (no header bar)
10
1 x full width image & text
placeholder (no header bar)
11
1 x full width image & text
placeholder (no header bar)
12
1 x full width image & text
placeholder (no header bar)
13
1 x full width image & text
placeholder (no header bar)
14
1 x full width image & text
placeholder (no header bar)
15
1 x full width image & text
placeholder (no header bar)
Find your black arrow
Automate QA Reports
16
Identify the enemy - Time
Several projects * 2 reports per month * 4 hours per report
17
Automate QA Reports
• External application
• Which gather data from sources like JIRA, SonarQube, QMetry
• And generate a ...
The Battle of Five Army
Challenges
19
Test Status
Test Case coverage and execution status
vs
Session Based Testing
20
Transparency
Internal Quality
vs
External Quality
21
Clients’ tooling
Different tools
The same tools, different security policies
22
Maintenance
One more internal tool for maintenance
JIRA / Confluence / QMetry API changes as example
23
Priority decreasing
Reports after release
Still required some manual work (about 2h)
More and more CD projects
24
And back again
Few years later
25
Adulthood
• Initial objectives are not valid anymore
• Experienced team
• Which know what is important for us
• Which gain...
Be reliable, do not measure, just do it
• Test Automation coverage dilemma
• DoD ensure me that I automated necessary test...
Now
• Project’s team makes decision if QA Report is needed and choose
metrics
• Most popular case: only final QA Report be...
The Ring / QA Reports
• Transform successfully
• Learn how to deliver bigger projects for demanding clients
• Identify wha...
And Back Again
30
Thank You
Upcoming SlideShare
Loading in …5
×

There and back again, Our journey with QA Reports and metrics

328 views

Published on

Presentation for Quality Excites 2017.
A company’s grow, as well as new and bigger clients who brought more difficult projects has begun transformation from start up into medium size and mature organization. How did the Cognifide survive that change? One of the answers was quality metrics and QA Reports. The story has begun few years ago when the first project team had to choose quality metrics and create the first periodic report. Zbyszek will show an example report with the most important metrics. But again, Cognifide was still growing, new and bigger projects brought unplanned adventures. How did the project teams deal with them? Zbyszek can tell that they did not win all the battles. So where are they now? Does the solution that passed the exam during transformation is still valuable for a mature organization?

Published in: Software
  • Be the first to comment

There and back again, Our journey with QA Reports and metrics

  1. 1. There and back again Our journey with QA Reports and metrics 1 Zbyszek Moćkun
  2. 2. Intro 2
  3. 3. Agenda 3 01 The Beginning 02 An unexpected journey / Define QA Reports and metrics 03 Find your Black Arrow / Automate QA Reports 04 The battle of five army / Challenges 05 And back again / Few years later
  4. 4. The beginning Long long time ago 4
  5. 5. Bigger, bigger and bigger • New clients from pharmacy, insurance and banking • Which required more documentation and more formal processes • Do not confident with agile methodologies and exploratory testing approach • Project duration from 2 to 6 months • Fast growing company • A lot of newcomers which do not understand our ways of working • Inexpierence team 5
  6. 6. How to control several simultanuously projects? • Governance on company level • Deliver always high / expected quality • Support inexperienced team / share knowledge • Reporting projects status to Company Board • Compare projects • Identify good practices • Understand project complexity 6
  7. 7. An unexpected journey Define QA Reports and metrics 7
  8. 8. How to identify metrics? • Metrics should encourage to ask questions not give the answers • Measure the processes not people • Qualitative and Quantitave • Highlight important processes, activities, ways of working • Each release has quality report 8
  9. 9. 9 1 x full width image & text placeholder (no header bar)
  10. 10. 10 1 x full width image & text placeholder (no header bar)
  11. 11. 11 1 x full width image & text placeholder (no header bar)
  12. 12. 12 1 x full width image & text placeholder (no header bar)
  13. 13. 13 1 x full width image & text placeholder (no header bar)
  14. 14. 14 1 x full width image & text placeholder (no header bar)
  15. 15. 15 1 x full width image & text placeholder (no header bar)
  16. 16. Find your black arrow Automate QA Reports 16
  17. 17. Identify the enemy - Time Several projects * 2 reports per month * 4 hours per report 17
  18. 18. Automate QA Reports • External application • Which gather data from sources like JIRA, SonarQube, QMetry • And generate a report on Confluence 18
  19. 19. The Battle of Five Army Challenges 19
  20. 20. Test Status Test Case coverage and execution status vs Session Based Testing 20
  21. 21. Transparency Internal Quality vs External Quality 21
  22. 22. Clients’ tooling Different tools The same tools, different security policies 22
  23. 23. Maintenance One more internal tool for maintenance JIRA / Confluence / QMetry API changes as example 23
  24. 24. Priority decreasing Reports after release Still required some manual work (about 2h) More and more CD projects 24
  25. 25. And back again Few years later 25
  26. 26. Adulthood • Initial objectives are not valid anymore • Experienced team • Which know what is important for us • Which gained the clients and board trust • From governance to consulting • QA status as part of project reports 26
  27. 27. Be reliable, do not measure, just do it • Test Automation coverage dilemma • DoD ensure me that I automated necessary tests on different levels • Why I should measure the metric if I knew the result? 27
  28. 28. Now • Project’s team makes decision if QA Report is needed and choose metrics • Most popular case: only final QA Report before UAT (are we production ready?) • Measure process improvement • For the last proces change (shift left) we measured the improvement rate • Eg. as a result we measured that Internal Quality raised from 50% even to 90% acceptance rate 28
  29. 29. The Ring / QA Reports • Transform successfully • Learn how to deliver bigger projects for demanding clients • Identify what is important for us • Teach newcomers our ways of working • And we have a tool for next transformation 29
  30. 30. And Back Again 30 Thank You

×