A Novel View of Applying FMECA to Software Engineering

1,893 views

Published on

An ASQ Reliability Division webinar by Robert Stoddard

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,893
On SlideShare
0
From Embeds
0
Number of Embeds
109
Actions
Shares
0
Downloads
50
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • {}
  • A Novel View of Applying FMECA to Software Engineering

    1. 1. A Novel View of Applying FMECA to Software Engineering Robert W. Stoddard ©2013 ASQ & Presentation Stoddard http://reliabilitycalendar.org/webina rs/
    2. 2. ASQ Reliability Division English Webinar Series One of the monthly webinars on topics of interest to reliability engineers. To view recorded webinar (available to ASQ Reliability Division members only) visit asq.org/reliability To sign up for the free and available to anyone live webinars visit reliabilitycalendar.org and select English Webinars to find links to register for upcoming events http://reliabilitycalendar.org/webina rs/
    3. 3. ASQ RD Webinar, 10 October 2013 A Novel View of Applying FMECA to Software Engineering by Robert W. Stoddard © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 3
    4. 4. ASQ RD Webinar, 10 October 2013 Conceptual View of FMECA FMECA is a structured approach to anticipating what can fail and why FMECA is a proactive way to prevent issues from occurring FMECA is considered a mature approach that is implemented in CMMI High Maturity organizations (Causal Analysis & Resolution) FMECA may be thought of as a top-down approach that may be combined with other approaches – Fault tree analysis and other root cause methods may be instantiated within the FMECA process – The results of FMECA may be used, for example, to guide prevention activities, such as Poka-Yoke © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 4
    5. 5. ASQ RD Webinar, 10 October 2013 Original Roots of FMECA During the 1950’s, FMECA enjoyed initial popularity within the Aerospace, Nuclear Engineering and Reliability Engineering fields Considered an essential technique for life-critical applications During the 1990’s, use of FMECA notably expanded to other industries and domains © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 5
    6. 6. ASQ RD Webinar, 10 October 2013 Example Business Use and Benefits Kraft Foods http://www.mcp-cg.com/pdf/FMECA%20Case%20Study%20-%20Kraft%2 Training and Consulting Services Company http://www.fmeainfocentre.com/examples/f0503_chowdary2.pdf Biotech Manufacturing http://www.pharmamanufacturing.com/articles/2009/115.html Behavioral Health Unit attempt to reduce patient falls http://www.hcpro.com/QPS-29783-234/Case-study-Use-FMEA-topreventMedicine Prescription Error Reduction http://www.fmeainfocentre.com/guides/qp0803reiling.pdf Numerous Case Studies http://www.fmeainfocentre.com/fmea_abstracts_2006.htm © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 6
    7. 7. ASQ RD Webinar, 10 October 2013 Failure Modes and Effects Criticality Analysis A structured method of using a multi-disciplinary team to brainstorm and anticipate failure modes and associated potential root causes Basically, three scores are assessed: 1. Severity 2. likelihood of occurrence 3. likelihood of escaping to the customer A Risk Priority Number (RPN) is computed by multiplying the three scores. (Normally, scores greater than 200 warrant prevention and mitigation planning by the team) DFMECA = Design FMECA PFMECA = Process FMECA © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 7
    8. 8. ASQ RD Webinar, 10 October 2013 FMECA Flow Step 1: Process Step or Design Component Step 4: Id Root Cause(s) Step 7: Calculate RPN Step 2: Id Failure Mode(s) Step 5: Id Probability of Occurrence of Root Cause Step 3: Id Effect & Severity Step 6: Id Probability of Escape of Root Cause Step 8: Id Prevention & Mitigation Actions; recalculate RPN © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Step 9: Continue Tracking RPN > 200 Version 010 8
    9. 9. ASQ RD Webinar, 10 October 2013 Traditional vs Modified Use of FMECA Traditional Use Modified Use Originally applied to products at the macro level Now applied to sub components of products, and to processes Originally used only objective historical data to establish the three RPN component scores Now applied with a combination of available historical data but also heavy use of subjective expert opinion of the RPN component scores Used with additional scrutiny and columns to discuss things such as: why the severity is high, why the root cause was possible, and which processes enabled the escape Now, the additional items of information to the left are considered optional. However, they are recognized as valuable information that may be added to the FMECA © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 9
    10. 10. ASQ RD Webinar, 10 October 2013 General Comments on the RPN Fields FMECA Field Comments Likelihood of Occurrence This is normally a scale of 1…10. Although most organizations find it useful to create text definitions of each level, one should remember that this scale should be ratio rather than ordinal. The higher the score, the more likely the root cause will occur. Severity This is also normally a ratio scale of 1…10. The higher the score, the more severe the consequence of the failure mode. Likelihood of Escape This is also normally a ratio scale of 1…10. The higher the score, the more likely the root cause will escape to be felt by a user or customer. See Don Wheeler’s commentary: http://www.qualitydigest.com/inside/quality-insider-column/problems-risk-priority-numbers.html © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 10
    11. 11. ASQ RD Webinar, 10 October 2013 Additional Thoughts re Likelihood of Occurrence Often there may be little data to support a consensus of the likelihood of occurrence In these situations, a surrogate for likelihood often works well For example, likelihood of occurrence of a code issue may correspond to the McCabe unit cyclomatic complexity value. As such, a profile of the unit complexity values may be mapped to the 1..10 scale so that differentiation may occur. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 11
    12. 12. ASQ RD Webinar, 10 October 2013 Additional Thoughts re Likelihood of Escape - 1 Often there may be little data to support a consensus of the likelihood of escape In these situations, a simplistic calculation often works well For example, assume a root cause is in unit A code. The team considers how the root cause may be found in downstream quality checks or testing. Continuing the example, assume that a code inspection, unit test, integration test and system test can possibly catch the root cause (e.g. defect) The team should quickly reach consensus on the probability that the defect will escape each of the downstream checks or tests. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 12
    13. 13. ASQ RD Webinar, 10 October 2013 Additional Thoughts re Likelihood of Escape - 2 Let’s assume the following escape probabilities: 75% chance of escaping the code inspection 50% chance of escaping the unit test 20% chance of escaping the integration test 90% chance of escaping the system test So, 0.75 * 0.50 * 0.20 * 0.90 = 7% and the 0..100% probability scale for escaping through all downstream activities can then be mapped to the 1..10 scale © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 13
    14. 14. ASQ RD Webinar, 10 October 2013 Software Engineering Application Generally applied with the perspective of the software-only content of a product May be driven by a product level FMECA or performed in isolation May be applied to different levels of abstraction of the software – Requirements – Architecture – Design – Code Remains a key tool to make software products and components more robust of “rainy day” operation of the software Provides a rich input to the software testing activity © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 14
    15. 15. ASQ RD Webinar, 10 October 2013 Enlightened Approach to FMECA of SW FMECA is a structured approach to identify what can go wrong and why Software exists in many forms as it is developed Software begins as an abstract form of requirements and needs Software then is defined as an architecture Software then proceeds through high and low levels of design Software finally arrives in the form of actual code (source, object, executable image) As such, FMECA may be applied to any and all the different abstractions of software! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 15
    16. 16. ASQ RD Webinar, 10 October 2013 The V Model for Software Development FMECA results drives test cases SW Reqts SW System Test SW Architecture SW Performance Test SW High Level Integration Test High Level Design Detailed Design Code © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. SW Low Level Integration Test SW Unit Test Version 010 16
    17. 17. ASQ RD Webinar, 10 October 2013 DFMEA on the Requirements The distinct requirements and/or needs identified for the software © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 17
    18. 18. ASQ RD Webinar, 10 October 2013 DFMEA on the Requirements The distinct operational scenarios or use cases © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 18
    19. 19. ASQ RD Webinar, 10 October 2013 DFMEA on the Requirements The distinct quality and performance measures for the overall software © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 19
    20. 20. ASQ RD Webinar, 10 October 2013 Question 1 To what degree do you believe SFMECA could benefit your organization’s software requirements quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 20
    21. 21. ASQ RD Webinar, 10 October 2013 Applying DFMEA to Software Architecture Each View of the Software Architecture may be analyzed via FMECA Remember, a view is a stakeholder representation of the architecture structure FMECA enables a proactive approach to assessing potential weaknesses or vulnerabilities in the software architecture © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 21
    22. 22. ASQ RD Webinar, 10 October 2013 FMECA on the Logical View The discrete services that the software should provide users © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 22
    23. 23. ASQ RD Webinar, 10 October 2013 FMECA on the Process View The different possible concurrent and independent processes, programs, or networks that communicate with each other © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 23
    24. 24. ASQ RD Webinar, 10 October 2013 FMECA on the Development View The Development Parts of the Software which are assigned to developers to work on © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 24
    25. 25. ASQ RD Webinar, 10 October 2013 FMECA on the Physical View The mapping of the objects from the other views on the processing nodes of the system © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 25
    26. 26. ASQ RD Webinar, 10 October 2013 Using Software Architecture FMECA Results The results of FMECA can be used as follows: – To adjust software architecture tactics or decisions – To re-assess the need for fault tolerance and error-handling – To guide the planning of software testing, especially robust software testing – To assess the risk of product shipment and warranty exposure The above actions may be decided based on: – The prevention and mitigation actions identified during the FMECA exercise – The RPN scores with values remaining above threshold determined by the team © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 26
    27. 27. ASQ RD Webinar, 10 October 2013 Question 2 To what degree do you believe SFMECA could benefit your organization’s software architecture quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 27
    28. 28. ASQ RD Webinar, 10 October 2013 Applying DFMEA to Software Design Each Depiction of the Software Design may be analyzed via FMECA Possible Software Design Depictions include: – Software Call Tree – Object Relationship Diagram – Entity Relationship Diagram, including all major interfaces – Control flow chart – Data flow chart – McCabe logical path diagram (Cyclomatic, Module Design, Subtrees) – Detailed State-Transition-Diagram – Message Sequence Charts – Network Performance Diagrams, including Markov and Petri-Net Models © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 28
    29. 29. ASQ RD Webinar, 10 October 2013 Software Call Tree © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 29
    30. 30. ASQ RD Webinar, 10 October 2013 Object Relationship Diagram Each object has internal states, inputs, and outputs. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 30
    31. 31. ASQ RD Webinar, 10 October 2013 Entity Relationship Diagram External System HW 1 User SW SW 1 1 2 User SW 3 3 External System 2 © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. HW 2 User 2 Version 010 31
    32. 32. ASQ RD Webinar, 10 October 2013 Control Flow Diagram Objects or Code Processes with control operations between each. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 32
    33. 33. ASQ RD Webinar, 10 October 2013 Data Flow Diagram Data Store 1 Objects or Code Processes sending different data to each other. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Data Store 2 Version 010 33
    34. 34. ASQ RD Webinar, 10 October 2013 McCabe Logical Path Cyclomatic Complexity = # Unique logical paths Module design complexity = # unique logical paths related to code design interfaces vv © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 34
    35. 35. ASQ RD Webinar, 10 October 2013 State-Transition Diagram State 2 State 5 State 4 State 1 State 3 Different events cause transitions from one particular state to another state. © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 35
    36. 36. ASQ RD Webinar, 10 October 2013 Message Sequence Chart © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 36
    37. 37. ASQ RD Webinar, 10 October 2013 Network Performance Diagrams Using simulation, such as Discrete Event Simulation, Throughput, cycle times, waste times, queue lengths are determined © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 37
    38. 38. ASQ RD Webinar, 10 October 2013 Question 3 To what degree do you believe SFMECA could benefit your organization’s software design quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 38
    39. 39. ASQ RD Webinar, 10 October 2013 Code Inputs and Outputs Analyze the outputs of the code units for deficient conditions © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 39
    40. 40. ASQ RD Webinar, 10 October 2013 Code External Interfaces Analyze all or a subset of external interfaces for proper operation Deficiencies in an external call can be failure modes © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 40
    41. 41. ASQ RD Webinar, 10 October 2013 Code Constants, Variables, Pointers Constants may be set or reset incorrectly Variables can go out of allowable range Pointers may be set incorrectly or not released when done (e.g. memory leak) Obviously only a critical subset of the above items to be analyzed! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 41
    42. 42. ASQ RD Webinar, 10 October 2013 Code Memory Utilization Memory usage may be analyzed Reference memory utilization patterns and anti-patterns © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 42
    43. 43. ASQ RD Webinar, 10 October 2013 Code Error Handling Look at existing error handling code Identify all exceptions to be handled Question missing exceptions Question how exceptions are to be handled © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 43
    44. 44. ASQ RD Webinar, 10 October 2013 Question 4 To what degree do you believe SFMECA could benefit your organization’s code quality? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 44
    45. 45. ASQ RD Webinar, 10 October 2013 Approach to Populating FMECA Template Teams have found it less distracting if they populate the FMECA template column by column This approach keeps focus on one aspect at a time The team can then more early assess the total effort needed to finish the FMECA Easier to have stops and starts if the FMECA session needs to run across different face-to-face meetings © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 45
    46. 46. ASQ RD Webinar, 10 October 2013 Approach to Reporting Completed FMECA Teams have found it more communicative to report the results of the FMECA row by row Teams will also reorder the rows in decreasing RPN magnitude Teams may also keep groups of rows that make logical sense to discuss together © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 46
    47. 47. ASQ RD Webinar, 10 October 2013 Relationship to Test Activity As discussed earlier, FMECA may be an excellent targeting mechanism for stress and robustness testing! Test staff have been found to have a rich perspective (detective, critical, search for ways to break things) that makes them superior facilitators of FMECAs FMECAs held during development provide a rewarding mechanism for development and test staff to talk, instead of the development followed by throwing the code over the wall to the testers. The prevention and mitigation thinking promoted by FMECA during software development can save development engineers a lot of time and rework by getting it right the first time © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 47
    48. 48. ASQ RD Webinar, 10 October 2013 Who Should Facilitate FMECA? Someone other than the author of the artifact under analysis The more independent the better! Sometimes, too much domain knowledge can hinder great facilitation (too easy for facilitator to get wrapped up in the answers instead of asking the probing questions) Test staff are excellent! Quality and process improvement staff can perform well if they have some knowledge of the domain and application Management and supervisors should not facilitate the FMECAs of work products of their subordinates! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 48
    49. 49. ASQ RD Webinar, 10 October 2013 Change from Peer Reviews & Inspections Peer Reviews & Formal Inspections FMECAs Require and depend on significant advance individual preparation Require less advance preparation by participants Focus on established standards and conventions from a compliance standpoint (Closed Focus) Focus on brainstormed failure modes (Open focus) Focus only on identifying noncompliances (defects) Focus on failure modes and corresponding root causes Only looks at actual issues or defects (e.g. they have already occurred) Purposely anticipates issues and defects that have yet to occur © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 49
    50. 50. ASQ RD Webinar, 10 October 2013 Patience is Required! FMECAs can be similar to diving for pearls! Divers know that they must accomplish 20 dives to find a decent pearl. Likewise, FMECAs may have to explore 20-30 items before a valuable recognition of a potential issue is found. Participants in FMECAs must have patience! They may feel like they are being peppered with a lengthy list of silly questions. Management must enable the resources and time to conduct FMECAs; Show data reflecting avoidance of rework and downstream issues! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 50
    51. 51. ASQ RD Webinar, 10 October 2013 Root Cause vs Underlying Causes FMECA teams can become mired in root cause discussions. If so, the team facilitator should accept the identification of an underlying cause. Root cause is the real source of the problem Underlying cause is the proximate cause of the problem but may not necessarily be the real root cause We don’t want the FMECA to expend unreasonable time and resources © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 51
    52. 52. ASQ RD Webinar, 10 October 2013 Tools to Use for FMECA Many tools exist to support FMECA. Free tools and tools for purchase may be identified with a simple internet search Many companies are satisfied to use a simple Excel spreadsheet Others desire a database approach to enable the sharing of data and results across the organization and/or product lines © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 52
    53. 53. ASQ RD Webinar, 10 October 2013 Managing Complexity and Exploding FMECAs FMECAs must be scoped to meet the needs of the business or organization Scoping must first address which products to be analyzed (HW, SW, management artifacts) Scoping must also address perceived risk. It might be reasoned that only certain parts of the product may be analyzed. For the parts identified, it may be desired to still drive deep in the FMECA to ensure critical aspects are fully reviewed. Most organizations find an incremental approach of FMECAs during product development the most beneficial. However, FMECAs can still be performed on completed products if one desires a rich risk assessment! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 53
    54. 54. ASQ RD Webinar, 10 October 2013 Who Should be on the FMECA Team? The FMECA team should be as multi perspective as possible If done at the product level, all engineering disciplines should participate If done within just software engineering, then different software perspectives should participate as follows: – – – – – – – Software Req’ts analyst Software Architecture and/or System engineer Software Design Software Coding (programmer) Software Test (all levels of test should participate) Software technology experts and domain experts Software process and quality staff Great facilitators with a detective mindset are needed! Facilitators need to manage both convergent and divergent thinking styles. Lastly, facilitators must manage time and patience levels! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 54
    55. 55. ASQ RD Webinar, 10 October 2013 Causing Active Institutional Learning Cycles! Many organizations have found more advanced and clever uses of FMECA! These organizations will use historical results of failure modes and corresponding root causes to pre-populate FMECA templates. In this approach, different templates are created based on the phase of the software lifecycle and nature of the artifact under analysis. With this approach, individual teams will assess themselves against the historical failure modes and root causes before venturing into the normal process of evaluating their artifacts. This helps ensure cycles of learning within the organization! This approach has been shown to be superior to large databases of lessons learned that are supposed to be searched by very busy development engineers! © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 55
    56. 56. ASQ RD Webinar, 10 October 2013 Question 5 Would you be interested in a ½ day to a full day tutorial consisting of hands-on practice using Software FMECA? © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 56
    57. 57. ASQ RD Webinar, 10 October 2013 Summary FMECA is simple, practical and hopefully easy FMECA is more rewarding than traditional peer reviews and inspections FMECA doesn’t require a tool investment – but rather some team training FMECA can literally be applied to almost anything to anticipate, prevent and mitigate issues © 2013 Six Sigma IDS, LLC. Confidential and Proprietary. Version 010 57

    ×