3. The Many Faces of Software Unreliability Taz Daughtrey Data and Analysis Center for Software James Madison University © 2011 Quanterion Solutions, Inc.

4. The Many Faces of Software Unreliability ASQ Reliability Division Webinar January 12, 2012 Defining software reliability Measuring software reliability Achieving software reliability The future

5. Errors => Defects => Failures

6. The Many Faces of Software Unreliability Defining software reliability

7. Reliability : does what is expected

8. Unreliability : doesn ’t do what is expected

9. unavailable compromised unsafe incorrect

18. The Many Faces of Software Unreliability Measuring software reliability

19. Reliability : measured in… … success/failure probability … M ean T ime T o F ailure

20. M ean T ime T o R epair mission failure $ loss risk exposure measured in …

24. R O I = return investment R O R I = risk exposure reduction reliability investment

25. Risk Avoidance costs $3 x 10 3 4 x 10 -4 $4 x 10 3  risk exposure reduction of $10 4 -$4 x 10 3 = $6 x 10 3

26. Risk Avoidance costs $3 x 10 3 4 x 10 -4 $4 x 10 3  risk exposure reduction of $10 4 -$4 x 10 3 = $6 x 10 3 Return On Risk Exposure Reduction =$6 x 10 3 / $3 x 10 3 = 2.0

27. Risk Mitigation costs $5 x 10 3 10 -3 Probability of occurrence $3 x 10 6 $3 x 10 3  risk exposure reduction of $10 4 -$3 x 10 3 = $7 x 10 3

28. Risk Mitigation costs $5 x 10 3 10 -3 Probability of occurrence $3 x 10 6 $3 x 10 3  risk exposure reduction of $10 4 -$3 x 10 3 = $7 x 10 3 Return On Risk Exposure Reduction =$7 x 10 3 / $5 x 10 3 = 1.4

29. The Many Faces of Software Unreliability Achieving software reliability

31. Software Reliability Engineering E Establish quantitative reliability targets Construct usage profiles of operational system Test statistically to predict system reliability

32. Software Reliability Estimation + Reliability Growth Modeling

33. Lifecycle Software Reliability Measurements

34. S tatistical M odeling and E stimation of R eliability F unctions for S oftware

35. Set measureable targets Design, implement Conduct appraisals Release? Rework?

36. reviews tests stakeholder agreement operational profiles verifiable requirements fault-tolerant design

37. FAULT ENCOUNTER FAULT SOURCE intentional accidental intentional accidental

38. FAULT ENCOUNTER FAULT SOURCE intentional accidental intentional accidental malicious tradeoffs

39. Attack + Countermeasure Tree

40. The Many Faces of Software Unreliability The future of software reliability

41. DACS Software Reliability Initiative = “Roadmap to Dependability”

42. DACS projected products: Handbook series Software Reliability and Security Testing Software Reliability and Security Specification Software Reliability and Security Assessment and Improvement Training offerings Software Reliability and Security Fundamentals + industry-specific modules Research support

44. ongoing mentoring on-the-job application management-sponsored project initial class session follow-up sessions

45. The Many Faces of Software Unreliability Taz Daughtrey [email_address] www.thedacs.com