Measurement and Control for Risk-based Test Cases and Activities Ellen Souza , Cristine Gusmão,  Keldjan Alves, Júlio Venâ...
Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul>...
Risk-based Testing - RBT Activities <ul><li>For each software requirement, risks are  identified  and  analyzed  (prioriti...
Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul>...
Case Study <ul><li>The  Health-Watcher  system requirements document was used to identify and analyze the risk related to ...
Case Study
Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul>...
Risk-based Testing Metrics <ul><li>Control and Measure the Risk-based Test Cases </li></ul><ul><li>Related to test cases a...
Risk-based Testing Metrics Identify Risks Activities M3.1  Number of risks per categories / Total number of risks Q3.1 . H...
Risk-based Testing Metrics Identify Risks Activities M11.1  (Sum (analysis duration of a requirement / Lines of a requirem...
Risk-based Testing Metrics Analyze Risks Activities M6.3.  Time spent to analyze all requirements / Number of requirements...
Risk-based Testing Metrics Analyze Risks Activities M14.1  Number of test cases with the same risk exposure / Total number...
Risk-based Testing Metrics Planning Tests Activities <ul><li>When : decide to test first, the requirements with High prior...
Risk-based Testing Metrics Design Tests Activities 35 24 38 Total No. Risks = 0 0 1 Low FR14  No. Risks = 2 2 7 Low FR13 N...
Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul>...
Conclusion and Future Work <ul><li>Metrics to control and measure the impact and advantages of RBT adoption in an organiza...
Measurement and Control for Risk-based Test Cases and Activities Ellen Souza , Cristine Gusmão,  Keldjan Alves, Júlio Venâ...
Upcoming SlideShare
Loading in …5
×

Measurement Control Risk Based Test Cases Activities Latw09

544 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
544
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Measurement Control Risk Based Test Cases Activities Latw09

  1. 1. Measurement and Control for Risk-based Test Cases and Activities Ellen Souza , Cristine Gusmão, Keldjan Alves, Júlio Venâncio and Renata Melo Department of Systems and Computing at Pernambuco University Rua Benfica, 455, Madalena, 50750-410, Recife – PE, Brasil pma.dsc.upe.br
  2. 2. Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul><ul><ul><li>RBT Process </li></ul></ul><ul><li>Risk-based Testing Metrics </li></ul><ul><ul><li>Control and Measure the Risk-based Test Cases </li></ul></ul><ul><ul><li>Control and Measure RBT- Activities </li></ul></ul><ul><li>Conclusion and Future Work </li></ul>
  3. 3. Risk-based Testing - RBT Activities <ul><li>For each software requirement, risks are identified and analyzed (prioritized). </li></ul><ul><li>Test cases are designed to assess the risk factors events. </li></ul><ul><li>When those tests are executed and a fail can be observed, testers report the defect so it can be fixed and the risk, related to the test case, mitigated ( controlled ). </li></ul>
  4. 4. Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul><ul><ul><li>RBTProcess </li></ul></ul><ul><li>Risk-based Testing Metrics </li></ul><ul><ul><li>Control and Measure the Risk-based Test Cases </li></ul></ul><ul><ul><li>Control and Measure RBT- Activities </li></ul></ul><ul><li>Conclusion and Future Work </li></ul>
  5. 5. Case Study <ul><li>The Health-Watcher system requirements document was used to identify and analyze the risk related to its requirements. The document contains nine functional requirements and nine non-functional requirements. </li></ul><ul><li>Three undergraduate students, with testing basic knowledge, took part as volunteers executing the RBTProcess activities and collecting the time spent to realize each one. </li></ul>
  6. 6. Case Study
  7. 7. Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul><ul><ul><li>RBTProcess </li></ul></ul><ul><li>Risk-based Testing Metrics </li></ul><ul><ul><li>Control and Measure the Risk-based Test Cases </li></ul></ul><ul><ul><li>Control and Measure RBT- Activities </li></ul></ul><ul><li>Conclusion and Future Work </li></ul>
  8. 8. Risk-based Testing Metrics <ul><li>Control and Measure the Risk-based Test Cases </li></ul><ul><li>Related to test cases and risks control </li></ul><ul><li>Control and Measure RBT- Activities </li></ul><ul><li>Evaluates the RBT activities by controlling the needed efforts and resources </li></ul><ul><li>The Goal Question Metric (GQM) approach, was used to define the proposed metrics </li></ul><ul><li>Gn  measure objective/goal </li></ul><ul><li>Qn.m  the questions that help us to understand the objective/goal </li></ul><ul><li>Mn.m  the metric that we have to collect to answer the questions </li></ul>
  9. 9. Risk-based Testing Metrics Identify Risks Activities M3.1 Number of risks per categories / Total number of risks Q3.1 . How many risks have been identified per categories? G3 . To know the risk categories that presents more risk factors. The risks can be classified according to some categories or taxonomies. Stability: 11 / 24 = 45% Completeness: 7 / 24 = 29% Clarity: 3 / 24 = 12% Validity: 3 / 24 = 12% M3.1
  10. 10. Risk-based Testing Metrics Identify Risks Activities M11.1 (Sum (analysis duration of a requirement / Lines of a requirement)) / Number of requirements Q11.1 . What is the mean time to analyze a line of a requirement? G11 . To know the average time spent to analyze a requirement with a certain number of lines. 24 / 3 (volunteers) = 8 risks M12.2 24 risks per meeting M12.1 6.35 minutes per requirement with  40 lines. M11.1 M12.1 Number of risks / Number of meetings Q12.1 . How many risks are found per meeting? G12 . To verify how productive the risk identification meetings are, in a quantitative manner.
  11. 11. Risk-based Testing Metrics Analyze Risks Activities M6.3. Time spent to analyze all requirements / Number of requirements Q6.3. How much time is spent on meetings to analyze relevant risks per requirement? M6.2. Compare the value of M4.1 with a threshold. Q6.2. Is the time spent on brainstorm meetings worthwhile? M6.1 (Number of identified risks / Number of requirements) / Meeting duration Q6.1 . How much time is spent on meetings to identify relevant risks per requirement? G6 . To support planning by providing effort estimations.  2.5 minutes per requirement M6.3
  12. 12. Risk-based Testing Metrics Analyze Risks Activities M14.1 Number of test cases with the same risk exposure / Total number o test cases Q14.1 . How many test cases shared the same risk exposure? G14 . Same Risk Exposure Identification 4 / 9 = 44 % M14.1b 2 / 9 = 22 % M14.1a 1.85 FR02. Specify complaint 1.75 FR11. Register tables 1.69 FR15. Update health unit 1.56 FR10. Login 1.38 FR12. Update complaint 1.38 FR14. Update employee 1.38 FR13. Register new employee 1.38 FR01. Query information 1.19 FR16. Change logged employee Risk Exposure Requirement/Feature
  13. 13. Risk-based Testing Metrics Planning Tests Activities <ul><li>When : decide to test first, the requirements with High priority; second, the requirements with Medium priority and third, the ones with Low priority. </li></ul><ul><li>What : perform also functional test for the requirements with High priority. </li></ul><ul><li>How : decide, for example, to perform white-box or structural testing for requirements with High priority. </li></ul><ul><li>Also automate test cases for a risk category. </li></ul>High 1.40 FR01. Query information High 1.19 FR15. Update health unit Medium 1.13 FR12. Update complaint Medium 1.13 FR02. Specify complaint Medium 1.07 FR11. Register tables Medium 1.00 FR16. Change logged employee Medium 1.00 FR10. Login Low 0.83 FR13. Register new employee Low 0.75 FR14. Update employee Priority RE Requirement/Feature
  14. 14. Risk-based Testing Metrics Design Tests Activities 35 24 38 Total No. Risks = 0 0 1 Low FR14 No. Risks = 2 2 7 Low FR13 No. Risks = 3 3 1 Medium FR16 No. Risks = 4 4 5 Medium FR10 No. Risks = 4 4 1 Medium FR11 No. Risks = 4 4 6 Medium FR12 No. Risks = 5 5 6 Medium FR02 6 + 1 = 7 1 6 High FR15 5 + 1 = 6 1 5 High FR01 No. Functional and RBT designed No. RBT No. Flows Prior. Feat.
  15. 15. Agenda <ul><li>Risk-based Testing </li></ul><ul><li>Case Study </li></ul><ul><ul><li>Health-Watcher system </li></ul></ul><ul><ul><li>RBTProcess </li></ul></ul><ul><li>Risk-based Testing Metrics </li></ul><ul><ul><li>Control and Measure the Risk-based Test Cases </li></ul></ul><ul><ul><li>Control and Measure RBT- Activities </li></ul></ul><ul><li>Conclusion and Future Work </li></ul>
  16. 16. Conclusion and Future Work <ul><li>Metrics to control and measure the impact and advantages of RBT adoption in an organization need to be defined and evaluated; </li></ul><ul><li>This work is an ongoing project: mechanisms for metrics collection and adaptation need to be defined and evaluated; </li></ul><ul><li>Tool ( RBTTool ) for collecting and analyzing data that can be used to evaluate and improve the suggested test cases based on identifying risk and to plot the tendency of evolution of the risks. </li></ul>
  17. 17. Measurement and Control for Risk-based Test Cases and Activities Ellen Souza , Cristine Gusmão, Keldjan Alves, Júlio Venâncio and Renata Melo Department of Systems and Computing at Pernambuco University Rua Benfica, 455, Madalena, 50750-410, Recife – PE, Brasil pma.dsc.upe.br

×