Producing Testable Requirements
 Piers Chamberlain piersc@intergen.co.nz &
 Robert Nugent robertn@intergen.co.nz
Producing Testable Requirements




 Requirements Inspection
 THE PROBLEM
 What the project team built wasn’t what I asked...
Producing Testable Requirements




 Quality Requirements
 The idea is to get your specification as ship-shape as possible...
Producing Testable Requirements




 What is a Testable Requirement
        Testable (in principal) means
                ...
Producing Testable Requirements




 Validation through Test Design
        Define a set of test cases to prove a requirem...
Producing Testable Requirements




 An Example
 Change Password
    The system will allow a user to change both their
   ...
Producing Testable Requirements




 Ambiguous…
 Change Password
    The system will allow a user to change both their
   ...
Producing Testable Requirements




 Inconsistent…
 Change Password
    The system will allow a user to change both their
...
Producing Testable Requirements




 Imprecise…
 Change Password
    The system will allow a user to change both their
   ...
Producing Testable Requirements




 Incomplete…
 Change Password
    The system will allow a user to change both their
  ...
Producing Testable Requirements




 Let‟s assume some validation rules…
 The system will validate a user's password
     ...
Producing Testable Requirements




 Version 2
 1. Change Password
 1.1 A user must be currently authenticated to change t...
Producing Testable Requirements




 Why it works
        It forces a user-centric view of how the system should
        h...
Producing Testable Requirements




 Guidelines for Quality Requirements
        Keep sentences short and break up paragra...
Producing Testable Requirements



 Requirement in good shape … now
 what am I prioritising
        Requirements (source o...
Producing Testable Requirements



 Why Assess the Risks
        Not all requirements are the same
        I can choice wh...
Producing Testable Requirements




    Example – Methods*
                                                         RISK

...
Producing Testable Requirements




 Example – risk calc spreadsheet
                                                     ...
Producing Testable Requirements




 Prioritising the Requirements
        By Requirement Type
        Must Have, Should H...
Producing Testable Requirements




 Example –using risk spreadsheet
                                                     ...
Producing Testable Requirements




 Prioritising Wrap up
        To successfully prioritise requirements you need to:
   ...
Producing Testable Requirements




 Other Benefits
        From Requirements Inspection
                Higher First Rele...
Producing Testable Requirements




 Some Simple First Steps
        Evaluate Requirements
                Review for Test...
Producing Testable Requirements




 Questions & Discussion
Producing Testable Requirements




 References & Further Reading
 Generating Testable Requirements
      Methodology for ...
Producing Testable Requirements




 References & Further Reading
 Risk-Based Prioritisation
       Requirement Ranking – ...
Upcoming SlideShare
Loading in...5
×

Producing Testable Requirements

6,056

Published on

Details approaches to software testing, as presented at a Twilight session at Intergen.

Published in: Technology
1 Comment
3 Likes
Statistics
Notes
  • I shuttered every time you wrote "principal" but meant "principle"!!!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
6,056
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
208
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Producing Testable Requirements

  1. 1. Producing Testable Requirements Piers Chamberlain piersc@intergen.co.nz & Robert Nugent robertn@intergen.co.nz
  2. 2. Producing Testable Requirements Requirements Inspection THE PROBLEM What the project team built wasn’t what I asked for…and I can’t believe that no-one picked up these issues before I started looking at the system myself! …but was what you asked for what you meant! 1. Natural language is very poor for communicating precise detail. 2. Human beings are so good at “interpreting” language that we make assumptions without thinking.
  3. 3. Producing Testable Requirements Quality Requirements The idea is to get your specification as ship-shape as possible. Feasible Scope Necessary Correct Complete Function Consistent Verifiable Testability Modifiable Traceable Maintenance Prioritized
  4. 4. Producing Testable Requirements What is a Testable Requirement Testable (in principal) means Precise Unambiguous Absolute state – The requirement is either met or not met …but it needs not be tested – can be either untested or indirectly tested Indirectly testable in practice Cost or risk reasons may prevent “real” testing Simulations, other tests that can provide some confidence
  5. 5. Producing Testable Requirements Validation through Test Design Define a set of test cases to prove a requirement in principal If you have a large number of tests (> 5 or 6) the requirement is probably too vague Especially if the tests are of different types Test Design Rules You can assume any other requirements in the spec are met You cannot assume anything not in the spec
  6. 6. Producing Testable Requirements An Example Change Password The system will allow a user to change both their password and username, but the user needs to have logged in to do this. On updating, the old credentials will be discarded and the user's new one activated. The rules for password / email validation remain the same.
  7. 7. Producing Testable Requirements Ambiguous… Change Password The system will allow a user to change both their password and username, but the user needs to have logged in to do this. On updating, the old credentials will be discarded and the user's new one activated. The rules for password / email validation remain the same.
  8. 8. Producing Testable Requirements Inconsistent… Change Password The system will allow a user to change both their password and username, but the user needs to have logged in to do this. On updating, the old credentials will be discarded and the user's new one activated. The rules for password / email validation remain the same.
  9. 9. Producing Testable Requirements Imprecise… Change Password The system will allow a user to change both their password and username, but the user needs to have logged in to do this. On updating, the old credentials will be discarded and the user's new one activated. The rules for password / email validation remain the same.
  10. 10. Producing Testable Requirements Incomplete… Change Password The system will allow a user to change both their password and username, but the user needs to have logged in to do this. On updating, the old credentials will be discarded and the user's new one activated. The rules for password / email validation remain the same.
  11. 11. Producing Testable Requirements Let‟s assume some validation rules… The system will validate a user's password The password must be at least 6 and no more than 40 characters The password can consist only of standard alphanumeric characters (i.e. a-z, A-Z and 0-9) Until the password is validated, it cannot be saved or used to authenticate a user. The system will validate a user's username The system will validate that the username conforms to the RFC 2822 standard as an email address The system will verify that the username is valid by sending an activation code as a hyperlink to that destination The username is accepted as valid if the user clicks the activation code sent Until the username is activated, it cannot be used to authenticate a user.
  12. 12. Producing Testable Requirements Version 2 1. Change Password 1.1 A user must be currently authenticated to change their password 1.1.1 The new password must meet the standard password validation rules* 1.2. A user must be currently authenticated to change their username 1.2.1 The new username is subject to the standard username validation rules* 2. Authentication 2.1 To authenticate a user must supply a valid username and password credentials at the login screen 2.1.2 A user‟s authentication is expired after 30 minutes 2.1.3 A user‟s authentication is expired immediately if they have elected to logout
  13. 13. Producing Testable Requirements Why it works It forces a user-centric view of how the system should hang together Introduces workflow, temporal elements Checks for consistency and completeness Can highlight design considerations where they are either required (and missing) or unnecessary Can provide behavioural and psychological insights
  14. 14. Producing Testable Requirements Guidelines for Quality Requirements Keep sentences short and break up paragraphs Use the active voice Supply all required detail Granularity – The “smaller number of tests” principle Avoid aggregation (conjunctions like “AND” and “OR” can suggest combined requirements) Write at a consistent level of detail throughout the document
  15. 15. Producing Testable Requirements Requirement in good shape … now what am I prioritising Requirements (source of the tests) At the same level of extraction or type Need to determine a measure to use (I am using Risk)
  16. 16. Producing Testable Requirements Why Assess the Risks Not all requirements are the same I can choice what to test first Testing duration is NOT guaranteed How can I conduct an assessment Need to determine: The source of requirements What method What criteria to use Who should be involved
  17. 17. Producing Testable Requirements Example – Methods* RISK Usage frequency Quality Probability of failure Damage Functional Volume Damage / Loss of Use (cost of failure) Example – Criteria Changed areas Critical area New component Visible area Impacted user base Usage frequency Deadlines Complex areas Defect history *Risk Based Testing, Strategies for Prioritising Tests against Deadlines Hans Schaefer.
  18. 18. Producing Testable Requirements Example – risk calc spreadsheet Risk calculation schema Impact factors Probability factors Prob. of defect detection visibility user impact frequency deadline pressure new people complexity weights weights Risk Risks addressed 1 3 10 1 3 3 function A 1 1 2 1 5 5 1 744 performance 0 1 1 1 1 4 1 208 usability of x 5 1 2 3 1 5 1 588 … 2 2 2 2 2 5 1 644 … 1 1 1 1 1 2 1 140 1 2 1 2 1 1 1 136
  19. 19. Producing Testable Requirements Prioritising the Requirements By Requirement Type Must Have, Should Have, Nice to Have (I Wish) Core Discretionary 1 to 100 From two factor risk assessment Clustering First Helps By Function By Functional Area Core vs. Auxiliary By sprint, cycle, release By Architectural layer
  20. 20. Producing Testable Requirements Example –using risk spreadsheet Risk calculation schema Impact factors Probability factors Prob. of defect detection visibility user impact frequency deadline pressure new people complexity weights weights Risk Risks addressed 31 3 10 1 3 3 function A 1 1 2 1 5 5 1 744 Performance profile of … 0 1 1 1 1 4 1 208 usability of x 5 2 1 2 3 1 5 1 588 UC213.1 2 2 2 2 2 5 1 644 Password must have … 1 1 1 1 1 2 1 1 140 … 1 2 1 2 1 1 1 136 4
  21. 21. Producing Testable Requirements Prioritising Wrap up To successfully prioritise requirements you need to: Determine Risk assessment criteria Use one or two factor method Weight the results Rank by chosen method And you will have: Objective view of project deliverables (the Risk if not delivered) Assistance in test scope analysis Prioritising of test effort
  22. 22. Producing Testable Requirements Other Benefits From Requirements Inspection Higher First Release quality Lower Cost of software maintenance Improved team dynamics / education From RiskPrioritisation Objective estimates Defect clustering analysis Defect resolution order Business perspective progress reporting
  23. 23. Producing Testable Requirements Some Simple First Steps Evaluate Requirements Review for Testability Conduct a risk assessment Must have, Should have or Could have Prioritise the Results Group the requirements Test this one first … to test this one last
  24. 24. Producing Testable Requirements Questions & Discussion
  25. 25. Producing Testable Requirements References & Further Reading Generating Testable Requirements Methodology for Writing High Quality Requirement Specifications Linda Rosenberg September 24 1998 Writing Quality Requirements by Karl E. Wiegers Writing Testable and Code-able Requirements Murat Guvenc / Borland On-Track Requirements by Rodger Drabick Better Software Magazine May/Jun 1999 (Vol. 1, Issue 3)
  26. 26. Producing Testable Requirements References & Further Reading Risk-Based Prioritisation Requirement Ranking – NO, No, No to High, Medium or Low Posted April 25th, 2008 by Steven Davis Risk-Based Quality Management Posted April 24th, 2008 by Tracy Lynne Dedore Prioritizing Requirements Posted April 9th, 2008 by Patrick Walsh Theory and Practice of Risk-based Testing by Felix Redmill Published in Software Testing, Verification and Reliability, Vol. 15, No. 1, March 2005 *Risk Based Testing, Strategies for Prioritizing Tests against Deadlines by Hans Schaefer, Published in Methods & Tools Volume 13 number 4, Winter 2005. Exploring Risk-based Testing and Its Implications by Felix Redmill Published in Software Testing, Verification and Reliability, Vol. 14, No. 1, March 2004 Risk-based Test Planning During System Development by Felix Redmill Invited paper at KKIO 2004, Gdansk, Poland, 5-8 October 2004 *Risk Based Testing and Metrics by Stale Amland, presented at EuroSTAR „99, November 8 – 12, 1999 Barcelona, Spain
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×