Quality Assuarance KPI Evaluation Tool

1. QA EVALUATION
TOOL
- A KPIs based evaluation
- An Excel/Spreadsheets based tool

2. Overview
 Problems of QA Evaluation
 New Evaluation Pros vs. Cons
 Criteria and Calculation
 Using the tool
 Questions and Answer

3. Problems of QA Evaluation
 The evaluation criteria are unspecific.
 QA Managers’ evaluations are based on
emotion.
 QA Managers evaluate in short term instead of
the whole year of good performance.
 Testers complain that their evaluations are
not clear and unfair.

4. New Evaluation Pros vs. Cons
Pros
 Automated KPIs tool.
 Emotionless evaluation.
 Easy to follow-up.
 Easy to compare the real
performance.
 Fair for every tester, no
more demotivation.
 Easy to double check.
Cons
 More “paper work” for
QA Managers.
 Must follow and update
regularly.
 Must organize massive
contests
 Evaluate up to QA
Senior.

5. Criteria and
Calculation

6. Late at Work – 5%
 QA Tester is paid by real working hours.
 If a tester go to work late: 0%
 If a tester never go to work late: 5%
 “Late at Work” definition:
 Present in the office after 8:10 AM without notice.
 Take day/half-day off without notice or
explanation.

7. Support Others – 5%
 Based on 360 degree Feedback.
 The final score will be the average of all Feedbacks.
 Formula:
Support Others 360 Feedback Average x 5%

8. Working Attitude – 15%
 Based on 360 degree Feedback.
 The final score will be the average of all Feedbacks.
 Formula:
Working Attitude 360 Feedback Average x 15%

9. Need Review Rate – 10%
 Apply for QADB Junior only.
 For QADB Senior, use Team’s Bug Quality Average instead.
 Need Review formula:
[100% - (No. of Need Review/No. of Total Bug x 100%)] x 10%
 Team’s Bug Quality formula:
Team’s Bug Quality Average x 10%

10. Personal Bug Quality – 15%
 Based on the result of Bug Scanning Contest.
 This is a cross teams contest.
 The contest must be arranged each 02 months.
 Formula:
Bug Scanning Contest Result x 15%

11. Personal Bug Quantity – 15%
 This is a two-step calculation because:
 Each team has its own project/game and number of bugs.
 When tester takes a day off, another one has to support
and do his/her task.
 Duplicated bugs are not counted.
 Testing Time = Normal Working Time + Overtime Working
Time
 Bug Per Hour formula:
Tester’s Total Bug Caught/Team’s Testing Time Average
 Personal Bug Quantity formula:
Tester’s Bug Rate/Team’s Highest Bug Rate x 15%

12. Checklist Contest – 20%
 Based on the result of Android Checklist Contest.
 This is a cross teams contest.
 The contest must be arranged each 02 months.
 Formula:
Android Checklist Contest Result x 20%

13. Report Skill – 5%
 Give the score base on the ranges (Good, Above
Average, Average, Below Average, Bad)
 Using 360 Degree Feedback method.
 QA Supervisor/Lead evaluate the reports of each
tester.
 Formula:
Report Score x 5%

14. Workshop Quantity – 5%
 Workshop must be validated by Trainer Team.
 Encourage sharing knowledge/experience.
 Formula:
𝐏𝐫𝐞𝐬𝐞𝐧𝐭𝐞𝐫′ 𝐬 𝐍𝐨. 𝐨𝐟 𝐖𝐨𝐫𝐤𝐬𝐡𝐨𝐩𝐬
𝐓𝐞𝐚𝐦′ 𝐬 𝐇𝐢𝐠𝐡𝐞𝐬𝐭 𝐍𝐨. 𝐨𝐟 𝐖𝐨𝐫𝐤𝐬𝐡𝐨𝐩𝐬
× 𝟓%

15. Workshop Quality – 5%
 Trainer must join the workshop and give the Presenter a
score for:
 Presentation skills
 Quality of the workshop
 If Presenter organize more than 01 workshop, the final
score will be average of his scores.
 Formula:
Presenter’s Workshop Scores Average x 5%

16. Management Skills – 5%
 This score is a bonus based on 360 Degree Feedback.
 Only the main key get this bonus.
 QA Supervisor/Lead must register the name of the main
key at the beginning of Evaluation term.
 Formula:
Management Skill 360 Feedback Average x 5%

17. Day Off & Comments
 Day Off is not counted in Total Score of a tester.
 Comments is not counted in Total Score but it’s
mandatory.
 Comments should be in compact form.
 QA Supervisor/Lead can collect comments from team
members (via 360 Feedback emails).

18. Using The Tool

19. Enter Necessary Information
 Teamlist Sheet:
1. Replace <YourName> part with your real name.
2. Change the Start and Finish time of Evaluation
period.
3. Enter the correct information of each tester.
 Position means the real position of the tester.
 QADB Position means the position of the tester
on QADB

20. Enter Necessary Information
 Input Sheet:
 Enter necessary information follow instruction.
 Do not change the format of data.
 Remember that blank and 0 are very different.

21. 360 Degree Feedbacks
1. Feedback request should be given from QA
Supervisor/Lead to tester.
2. Each tester can feedback about all members in his
team (include himself).
3. Tester will give feedback follow the template.
4. QA Supervisor/Lead will sum-up and use the result for
the evaluation.
Rank Average
Good 81 - 100
Above Average 61 - 80
Average 41 - 60
Below Average 21 - 40
Bad 0 - 20

22. 360 Degree Feedbacks
 360 Feedback Template:
 Use “Team 360 Feedback” tool to sum-up.

23. 360 Degree Feedbacks
 QA Supervisor/Lead can have his/her own evaluation for
each tester.
 The evaluation of QA Supervisor/Lead must be another
360 Feedback email or in excel file.
 QA Supervisor/Lead point of view is treated equally with
tester’s.
 The Evaluation Tool and all 360 Feedback emails
(compressed in a zip file) must be send to all QAPMs as
attachment.

24. Understand The Result
 The results of each 02 months are automatically
calculated in Monthly-Points sheet.
 Final results of Evaluation Term (06 months) are calculated
in SUM-UP sheet.
 Total Sum-up Chart compare all the testers in team.
 To see how a tester improves his/her performance though
Evaluation Term, enter the name of the tester into cell
C58.

25. Questions &
Answers