New Testing Standards Are on the Horizon: What Will Be Their Impact?

253 views

Published on

The history of testing standards has not always been auspicious. Testing standards documents have been expensive to obtain, limited in scope, inflexible in expectations, and inconsistent. However, they contain important lessons learned from experienced practitioners—if a tester is willing to overcome the obstacles to get to the useful information. A set of new international standards is coming. These new standards are tailorable, consistent, and comprehensive in scope. In addition, they will be freely available (some are already). Claire Lohr provides a complete roadmap to all of the available—or soon-to-be-available—testing-related standards. Learn where to go for testing process guidelines, complete definitions of all test design techniques, full examples of test documentation (for both agile and traditional projects), and free international standards documents. Take away a “start-up guide” for how different types of projects can use the new standards along with valuable tips and practical lessons you can get from these standards.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
253
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

New Testing Standards Are on the Horizon: What Will Be Their Impact?

  1. 1. T24 Special Topics 5/2/2013 3:00:00 PM New Testing Standards Are on the Horizon: What Will Be Their Impact? Presented by: Claire Lohr Lohr Systems Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Claire Lohr Claire Lohr has been an active professional in the computer field for thirty years, with the last twenty years emphasizing software process improvement and testing. Claire provides training - design, authoring, and instruction - and consulting services for a wide variety of both government and commercial clients. She is a Lloyd’s Register trained ISO 9000 Lead Auditor and has been trained to perform Software Capability Evaluations for the SW-CMM. Claire was the chair of the Working Group for the IEEE Std 829-2008 Software and System Test Documentation, and has served on both the IEEE Computer Society’s Software and Systems Engineering Standards Committee and the IEEE Computer Society’s Standards Advisory Board.
  3. 3. 4/16/2013 New Testing Standards on the Horizon: What Will Be Their Impact? Claire L. Lohr Lohr Systems P.O. Box 2998 Reston, VA 20195 703.472.5457 clohr@computer.org 1 Topics • • • • • • • • Why bother with standards? Traditional objections What’s available (free first) Impact of ISO 29119 Examples How to start Additional resources ?’s 2 1
  4. 4. 4/16/2013 Why bother with standards? • Common problems have common solutions • Standards are reviewed and modified by groups of very experienced ($$$$) practitioners • You can’t search the Internet for information unless you know what it is commonly called (or if you have never heard of it) 3 Traditional Objections Past Inconsistent and incomplete Overkill Present Merging together Includes high integrity and lower integrity choices Just theoretical Includes examples Expensive Increasingly free Time consuming ROI of 1:5-6 4 2
  5. 5. 4/16/2013 What’s available: free 1. nist.gov Special Publication 800-115 Technical Guide to Information Security Testing and Assessment 2. open-stand.org Commitment for the future by IEEE, IETF, IAB, Internet Society and W3C. 5 What’s available: free 3. Certification bodies of knowledge • swebok.org Summary of testing (& rest of SW engineering) • pmi.org The Software Extension to the PMBOK® Guide • istqb.org Glossary of testing terms & multiple tester certifications • buildsecurityin.us-cert.gov Software Assurance (SwA) Competency Model 6 3
  6. 6. 4/16/2013 What’s available: free 4. Other free resources • http://pascal.computer.org/sev_display/ind ex.action SEVOCAB definitions • http://standards.iso.org/ittf/PubliclyAvailabl eStandards/index.html free ISO standards 7 What’s available: free (example) SWEBOK testing section Fundamentals Levels Techniques Measures Process Tools 8 4
  7. 7. 4/16/2013 What’s available: not free 7925-1 Vocabulary 7925-2 Component testing 12207 Software life cycle 15026 Integrity levels 25010 Quality requirements 25051 Quality reqm’ts for COTS 829 Doc’s 1008 Unit test 1012 V&V 1028 Reviews 1044 Defect categories 9 What’s coming 29119 Systems and software engineering—Software testing— • Part 1: Concepts and definitions • Part 2: Test processes • Part 3: Test documentation • Part 4: Test techniques 10 5
  8. 8. 4/16/2013 ISO 29119-1 Concepts and definitions 4. Definitions 5. Concepts 6. Approaches 7. Automation 8. Defects 11 ISO 29119-2 Test processes 6. Organizational 7. Management 8. Dynamic 8.1 Introduction 8.2 Design & implementation 8.3 Environment setup & maintenance 8.4 Incident reporting 12 6
  9. 9. 4/16/2013 ISO 29119-3 Test documentation 5. Organizational 5.2 Policy NEW 5.3 Strategy 6. Management 6.2 Plan 6.3 Status Report 6.4 Completion Report 13 ISO 29119-3 Test documentation 7. Dynamic 7.2 Design 7.3 Case 7.4 Procedure 7.5 Data requirements 7.6 Environment requirements 7.7 Data readiness report 7.8 Environment readiness report 7.9 Actual result 7.10 Test Result 7.11 Execution Log 7.12 Incident report NEW NEW NEW NEW 14 7
  10. 10. 4/16/2013 ISO 29119-4 Test techniques 5.2 • • • • • • • • • • • • Equivalence partitioning Classification tree Boundary value analysis Syntax testing Combinatorial Decision table Cause-effect graphing State transition Use case Scenario Error guessing Random 15 ISO 29119-4 Test techniques 5.3 • • • • • Statement Branch Decision Condition Data flow D 6. Coverage A B Annex B Specification-based examples E Annex C Structure-based examples 16 8
  11. 11. 4/16/2013 Impact of ISO 29119 Past Future Inconsistent and incomplete Overkill Just theoretical Expensive Time consuming Consistent and complete in one place Includes high integrity and lower integrity choices Includes examples Increasingly free ROI of 1:5-6 17 Example #1 I want to improve our testing processes 29119-3 5.3 Strategy (for focus) Test Manager • • • • • • Risk management Test selection and prioritization Test documentation and reporting Test automation and tools Configuration management Incident management 18 9
  12. 12. 4/16/2013 Example #1 (test manager) 29119-2 8.3 Incident reporting process a) Test results are analyzed b) New incidents are confirmed, if any c) New incident report details are created d) The status and details of previouslyraised incidents are determined e) Previously-raised incident report details are updated as appropriate f) New and/or updated incident reports are communicated to the relevant stakeholders 19 Example #1 (test manager) 29119-3 7.12 Incident report 1. 2. 3. 4. 5. 6. 7. 8. Timing information Add related Originator incidents Context Description of the incident Originator’s assessment of severity Originator’s assessment of priority Risk Status of the incident 20 10
  13. 13. 4/16/2013 Example #2 I want to document better so I can get another job Tester 29119-2 Annex A Example Test Design* 29119-3 Annex H Example Test Design * Chose this one – more complete and more summary 21 How to start Suggested steps for transition 1. Survey your current baseline 2. Choose most valuable “next steps” (Kaizen) 3. Implement a pilot 4. Share what works 5. Continue to improve 22 11
  14. 14. 4/16/2013 Additional Resources www.iso.org www.ieee.org www. global.ihs.com The Roadmap to Software Engineering Standards: a Standards Based Guide, James W. Moore, 2006 23 Questions? 24 12
  15. 15. 4/16/2013 The End 25 13

×