Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Novel View of Applying FMECA to Software Engineering

2,275 views

Published on

An ASQ Reliability Division webinar by Robert Stoddard

Published in: Technology, Business
  • Be the first to comment

A Novel View of Applying FMECA to Software Engineering

  1. 1. A Novel View of Applying FMECA to Software Engineering Robert W. Stoddard ©2013 ASQ & Presentation Stoddard http://reliabilitycalendar.org/webina rs/
  2. 2. ASQ Reliability Division English Webinar Series One of the monthly webinars on topics of interest to reliability engineers. To view recorded webinar (available to ASQ Reliability Division members only) visit asq.org/reliability To sign up for the free and available to anyone live webinars visit reliabilitycalendar.org and select English Webinars to find links to register for upcoming events http://reliabilitycalendar.org/webina rs/
  3. 3. ASQ RD Webinar, 10 October 2013 A Novel View of Applying FMECA to Software Engineering by Robert W. Stoddard © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 3
  4. 4. ASQ RD Webinar, 10 October 2013 Conceptual View of FMECA FMECA is a structured approach to anticipating what can fail and why FMECA is a proactive way to prevent issues from occurring FMECA is considered a mature approach that is implemented in CMMI High Maturity organizations (Causal Analysis & Resolution) FMECA may be thought of as a top-down approach that may be combined with other approaches – Fault tree analysis and other root cause methods may be instantiated within the FMECA process – The results of FMECA may be used, for example, to guide prevention activities, such as Poka-Yoke © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 4
  5. 5. ASQ RD Webinar, 10 October 2013 Original Roots of FMECA During the 1950’s, FMECA enjoyed initial popularity within the Aerospace, Nuclear Engineering and Reliability Engineering fields Considered an essential technique for life-critical applications During the 1990’s, use of FMECA notably expanded to other industries and domains © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 5
  6. 6. ASQ RD Webinar, 10 October 2013 Example Business Use and Benefits Kraft Foods http://www.mcp-cg.com/pdf/FMECA%20Case%20Study%20-%20Kraft%2 Training and Consulting Services Company http://www.fmeainfocentre.com/examples/f0503_chowdary2.pdf Biotech Manufacturing http://www.pharmamanufacturing.com/articles/2009/115.html Behavioral Health Unit attempt to reduce patient falls http://www.hcpro.com/QPS-29783-234/Case-study-Use-FMEA-topreventMedicine Prescription Error Reduction http://www.fmeainfocentre.com/guides/qp0803reiling.pdf Numerous Case Studies http://www.fmeainfocentre.com/fmea_abstracts_2006.htm © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 6
  7. 7. ASQ RD Webinar, 10 October 2013 Failure Modes and Effects Criticality Analysis A structured method of using a multi-disciplinary team to brainstorm and anticipate failure modes and associated potential root causes Basically, three scores are assessed: 1. Severity 2. likelihood of occurrence 3. likelihood of escaping to the customer A Risk Priority Number (RPN) is computed by multiplying the three scores. (Normally, scores greater than 200 warrant prevention and mitigation planning by the team) DFMECA = Design FMECA PFMECA = Process FMECA © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 7
  8. 8. ASQ RD Webinar, 10 October 2013 FMECA Flow Step 1: Process Step or Design Component Step 4: Id Root Cause(s) Step 7: Calculate RPN Step 2: Id Failure Mode(s) Step 5: Id Probability of Occurrence of Root Cause Step 3: Id Effect & Severity Step 6: Id Probability of Escape of Root Cause Step 8: Id Prevention & Mitigation Actions; recalculate RPN © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Step 9: Continue Tracking RPN > 200 Version 010 8
  9. 9. ASQ RD Webinar, 10 October 2013 Traditional vs Modified Use of FMECA Traditional Use Modified Use Originally applied to products at the macro level Now applied to sub components of products, and to processes Originally used only objective historical data to establish the three RPN component scores Now applied with a combination of available historical data but also heavy use of subjective expert opinion of the RPN component scores Used with additional scrutiny and columns to discuss things such as: why the severity is high, why the root cause was possible, and which processes enabled the escape Now, the additional items of information to the left are considered optional. However, they are recognized as valuable information that may be added to the FMECA © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 9
  10. 10. ASQ RD Webinar, 10 October 2013 General Comments on the RPN Fields FMECA Field Comments Likelihood of Occurrence This is normally a scale of 1…10. Although most organizations find it useful to create text definitions of each level, one should remember that this scale should be ratio rather than ordinal. The higher the score, the more likely the root cause will occur. Severity This is also normally a ratio scale of 1…10. The higher the score, the more severe the consequence of the failure mode. Likelihood of Escape This is also normally a ratio scale of 1…10. The higher the score, the more likely the root cause will escape to be felt by a user or customer. See Don Wheeler’s commentary: http://www.qualitydigest.com/inside/quality-insider-column/problems-risk-priority-numbers.html © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 10
  11. 11. ASQ RD Webinar, 10 October 2013 Additional Thoughts re Likelihood of Occurrence Often there may be little data to support a consensus of the likelihood of occurrence In these situations, a surrogate for likelihood often works well For example, likelihood of occurrence of a code issue may correspond to the McCabe unit cyclomatic complexity value. As such, a profile of the unit complexity values may be mapped to the 1..10 scale so that differentiation may occur. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 11
  12. 12. ASQ RD Webinar, 10 October 2013 Additional Thoughts re Likelihood of Escape - 1 Often there may be little data to support a consensus of the likelihood of escape In these situations, a simplistic calculation often works well For example, assume a root cause is in unit A code. The team considers how the root cause may be found in downstream quality checks or testing. Continuing the example, assume that a code inspection, unit test, integration test and system test can possibly catch the root cause (e.g. defect) The team should quickly reach consensus on the probability that the defect will escape each of the downstream checks or tests. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 12
  13. 13. ASQ RD Webinar, 10 October 2013 Additional Thoughts re Likelihood of Escape - 2 Let’s assume the following escape probabilities: 75% chance of escaping the code inspection 50% chance of escaping the unit test 20% chance of escaping the integration test 90% chance of escaping the system test So, 0.75 * 0.50 * 0.20 * 0.90 = 7% and the 0..100% probability scale for escaping through all downstream activities can then be mapped to the 1..10 scale © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 13
  14. 14. ASQ RD Webinar, 10 October 2013 Software Engineering Application Generally applied with the perspective of the software-only content of a product May be driven by a product level FMECA or performed in isolation May be applied to different levels of abstraction of the software – Requirements – Architecture – Design – Code Remains a key tool to make software products and components more robust of “rainy day” operation of the software Provides a rich input to the software testing activity © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 14
  15. 15. ASQ RD Webinar, 10 October 2013 Enlightened Approach to FMECA of SW FMECA is a structured approach to identify what can go wrong and why Software exists in many forms as it is developed Software begins as an abstract form of requirements and needs Software then is defined as an architecture Software then proceeds through high and low levels of design Software finally arrives in the form of actual code (source, object, executable image) As such, FMECA may be applied to any and all the different abstractions of software! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 15
  16. 16. ASQ RD Webinar, 10 October 2013 The V Model for Software Development FMECA results drives test cases SW Reqts SW System Test SW Architecture SW Performance Test SW High Level Integration Test High Level Design Detailed Design Code © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. SW Low Level Integration Test SW Unit Test Version 010 16
  17. 17. ASQ RD Webinar, 10 October 2013 DFMEA on the Requirements The distinct requirements and/or needs identified for the software © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 17
  18. 18. ASQ RD Webinar, 10 October 2013 DFMEA on the Requirements The distinct operational scenarios or use cases © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 18
  19. 19. ASQ RD Webinar, 10 October 2013 DFMEA on the Requirements The distinct quality and performance measures for the overall software © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 19
  20. 20. ASQ RD Webinar, 10 October 2013 Question 1 To what degree do you believe SFMECA could benefit your organization’s software requirements quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 20
  21. 21. ASQ RD Webinar, 10 October 2013 Applying DFMEA to Software Architecture Each View of the Software Architecture may be analyzed via FMECA Remember, a view is a stakeholder representation of the architecture structure FMECA enables a proactive approach to assessing potential weaknesses or vulnerabilities in the software architecture © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 21
  22. 22. ASQ RD Webinar, 10 October 2013 FMECA on the Logical View The discrete services that the software should provide users © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 22
  23. 23. ASQ RD Webinar, 10 October 2013 FMECA on the Process View The different possible concurrent and independent processes, programs, or networks that communicate with each other © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 23
  24. 24. ASQ RD Webinar, 10 October 2013 FMECA on the Development View The Development Parts of the Software which are assigned to developers to work on © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 24
  25. 25. ASQ RD Webinar, 10 October 2013 FMECA on the Physical View The mapping of the objects from the other views on the processing nodes of the system © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 25
  26. 26. ASQ RD Webinar, 10 October 2013 Using Software Architecture FMECA Results The results of FMECA can be used as follows: – To adjust software architecture tactics or decisions – To re-assess the need for fault tolerance and error-handling – To guide the planning of software testing, especially robust software testing – To assess the risk of product shipment and warranty exposure The above actions may be decided based on: – The prevention and mitigation actions identified during the FMECA exercise – The RPN scores with values remaining above threshold determined by the team © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 26
  27. 27. ASQ RD Webinar, 10 October 2013 Question 2 To what degree do you believe SFMECA could benefit your organization’s software architecture quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 27
  28. 28. ASQ RD Webinar, 10 October 2013 Applying DFMEA to Software Design Each Depiction of the Software Design may be analyzed via FMECA Possible Software Design Depictions include: – Software Call Tree – Object Relationship Diagram – Entity Relationship Diagram, including all major interfaces – Control flow chart – Data flow chart – McCabe logical path diagram (Cyclomatic, Module Design, Subtrees) – Detailed State-Transition-Diagram – Message Sequence Charts – Network Performance Diagrams, including Markov and Petri-Net Models © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 28
  29. 29. ASQ RD Webinar, 10 October 2013 Software Call Tree © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 29
  30. 30. ASQ RD Webinar, 10 October 2013 Object Relationship Diagram Each object has internal states, inputs, and outputs. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 30
  31. 31. ASQ RD Webinar, 10 October 2013 Entity Relationship Diagram External System HW 1 User SW SW 1 1 2 User SW 3 3 External System 2 © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. HW 2 User 2 Version 010 31
  32. 32. ASQ RD Webinar, 10 October 2013 Control Flow Diagram Objects or Code Processes with control operations between each. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 32
  33. 33. ASQ RD Webinar, 10 October 2013 Data Flow Diagram Data Store 1 Objects or Code Processes sending different data to each other. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Data Store 2 Version 010 33
  34. 34. ASQ RD Webinar, 10 October 2013 McCabe Logical Path Cyclomatic Complexity = # Unique logical paths Module design complexity = # unique logical paths related to code design interfaces vv © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 34
  35. 35. ASQ RD Webinar, 10 October 2013 State-Transition Diagram State 2 State 5 State 4 State 1 State 3 Different events cause transitions from one particular state to another state. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 35
  36. 36. ASQ RD Webinar, 10 October 2013 Message Sequence Chart © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 36
  37. 37. ASQ RD Webinar, 10 October 2013 Network Performance Diagrams Using simulation, such as Discrete Event Simulation, Throughput, cycle times, waste times, queue lengths are determined © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 37
  38. 38. ASQ RD Webinar, 10 October 2013 Question 3 To what degree do you believe SFMECA could benefit your organization’s software design quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 38
  39. 39. ASQ RD Webinar, 10 October 2013 Code Inputs and Outputs Analyze the outputs of the code units for deficient conditions © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 39
  40. 40. ASQ RD Webinar, 10 October 2013 Code External Interfaces Analyze all or a subset of external interfaces for proper operation Deficiencies in an external call can be failure modes © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 40
  41. 41. ASQ RD Webinar, 10 October 2013 Code Constants, Variables, Pointers Constants may be set or reset incorrectly Variables can go out of allowable range Pointers may be set incorrectly or not released when done (e.g. memory leak) Obviously only a critical subset of the above items to be analyzed! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 41
  42. 42. ASQ RD Webinar, 10 October 2013 Code Memory Utilization Memory usage may be analyzed Reference memory utilization patterns and anti-patterns © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 42
  43. 43. ASQ RD Webinar, 10 October 2013 Code Error Handling Look at existing error handling code Identify all exceptions to be handled Question missing exceptions Question how exceptions are to be handled © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 43
  44. 44. ASQ RD Webinar, 10 October 2013 Question 4 To what degree do you believe SFMECA could benefit your organization’s code quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 44
  45. 45. ASQ RD Webinar, 10 October 2013 Approach to Populating FMECA Template Teams have found it less distracting if they populate the FMECA template column by column This approach keeps focus on one aspect at a time The team can then more early assess the total effort needed to finish the FMECA Easier to have stops and starts if the FMECA session needs to run across different face-to-face meetings © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 45
  46. 46. ASQ RD Webinar, 10 October 2013 Approach to Reporting Completed FMECA Teams have found it more communicative to report the results of the FMECA row by row Teams will also reorder the rows in decreasing RPN magnitude Teams may also keep groups of rows that make logical sense to discuss together © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 46
  47. 47. ASQ RD Webinar, 10 October 2013 Relationship to Test Activity As discussed earlier, FMECA may be an excellent targeting mechanism for stress and robustness testing! Test staff have been found to have a rich perspective (detective, critical, search for ways to break things) that makes them superior facilitators of FMECAs FMECAs held during development provide a rewarding mechanism for development and test staff to talk, instead of the development followed by throwing the code over the wall to the testers. The prevention and mitigation thinking promoted by FMECA during software development can save development engineers a lot of time and rework by getting it right the first time © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 47
  48. 48. ASQ RD Webinar, 10 October 2013 Who Should Facilitate FMECA? Someone other than the author of the artifact under analysis The more independent the better! Sometimes, too much domain knowledge can hinder great facilitation (too easy for facilitator to get wrapped up in the answers instead of asking the probing questions) Test staff are excellent! Quality and process improvement staff can perform well if they have some knowledge of the domain and application Management and supervisors should not facilitate the FMECAs of work products of their subordinates! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 48
  49. 49. ASQ RD Webinar, 10 October 2013 Change from Peer Reviews & Inspections Peer Reviews & Formal Inspections FMECAs Require and depend on significant advance individual preparation Require less advance preparation by participants Focus on established standards and conventions from a compliance standpoint (Closed Focus) Focus on brainstormed failure modes (Open focus) Focus only on identifying noncompliances (defects) Focus on failure modes and corresponding root causes Only looks at actual issues or defects (e.g. they have already occurred) Purposely anticipates issues and defects that have yet to occur © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 49
  50. 50. ASQ RD Webinar, 10 October 2013 Patience is Required! FMECAs can be similar to diving for pearls! Divers know that they must accomplish 20 dives to find a decent pearl. Likewise, FMECAs may have to explore 20-30 items before a valuable recognition of a potential issue is found. Participants in FMECAs must have patience! They may feel like they are being peppered with a lengthy list of silly questions. Management must enable the resources and time to conduct FMECAs; Show data reflecting avoidance of rework and downstream issues! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 50
  51. 51. ASQ RD Webinar, 10 October 2013 Root Cause vs Underlying Causes FMECA teams can become mired in root cause discussions. If so, the team facilitator should accept the identification of an underlying cause. Root cause is the real source of the problem Underlying cause is the proximate cause of the problem but may not necessarily be the real root cause We don’t want the FMECA to expend unreasonable time and resources © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 51
  52. 52. ASQ RD Webinar, 10 October 2013 Tools to Use for FMECA Many tools exist to support FMECA. Free tools and tools for purchase may be identified with a simple internet search Many companies are satisfied to use a simple Excel spreadsheet Others desire a database approach to enable the sharing of data and results across the organization and/or product lines © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 52
  53. 53. ASQ RD Webinar, 10 October 2013 Managing Complexity and Exploding FMECAs FMECAs must be scoped to meet the needs of the business or organization Scoping must first address which products to be analyzed (HW, SW, management artifacts) Scoping must also address perceived risk. It might be reasoned that only certain parts of the product may be analyzed. For the parts identified, it may be desired to still drive deep in the FMECA to ensure critical aspects are fully reviewed. Most organizations find an incremental approach of FMECAs during product development the most beneficial. However, FMECAs can still be performed on completed products if one desires a rich risk assessment! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 53
  54. 54. ASQ RD Webinar, 10 October 2013 Who Should be on the FMECA Team? The FMECA team should be as multi perspective as possible If done at the product level, all engineering disciplines should participate If done within just software engineering, then different software perspectives should participate as follows: – – – – – – – Software Req’ts analyst Software Architecture and/or System engineer Software Design Software Coding (programmer) Software Test (all levels of test should participate) Software technology experts and domain experts Software process and quality staff Great facilitators with a detective mindset are needed! Facilitators need to manage both convergent and divergent thinking styles. Lastly, facilitators must manage time and patience levels! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 54
  55. 55. ASQ RD Webinar, 10 October 2013 Causing Active Institutional Learning Cycles! Many organizations have found more advanced and clever uses of FMECA! These organizations will use historical results of failure modes and corresponding root causes to pre-populate FMECA templates. In this approach, different templates are created based on the phase of the software lifecycle and nature of the artifact under analysis. With this approach, individual teams will assess themselves against the historical failure modes and root causes before venturing into the normal process of evaluating their artifacts. This helps ensure cycles of learning within the organization! This approach has been shown to be superior to large databases of lessons learned that are supposed to be searched by very busy development engineers! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 55
  56. 56. ASQ RD Webinar, 10 October 2013 Question 5 Would you be interested in a ½ day to a full day tutorial consisting of hands-on practice using Software FMECA? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 56
  57. 57. ASQ RD Webinar, 10 October 2013 Summary FMECA is simple, practical and hopefully easy FMECA is more rewarding than traditional peer reviews and inspections FMECA doesn’t require a tool investment – but rather some team training FMECA can literally be applied to almost anything to anticipate, prevent and mitigate issues © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 57

×