West                                                                Virginia                                              ...
West                                         Virginia                                        University                  O...
Automated Risk                                                  West               Assessment                             ...
Automated Risk Assessment                               (continued)                                                       ...
West               Automated Risk Assessment                     Virginia         Architectural-Level Risk Assessment     ...
West              Automated Risk Assessment                                                     (continued)               ...
Automated Risk Assessment                         West                                                          Virginia  ...
Automated Risk Assessment                      West                                                           Virginia    ...
West                                                                                                                     V...
Example: Pacemaker               West                                 Virginia                                University1)...
West             Case Study: Pacemaker      (continued)                                                                   ...
West              Case Study: Pacemaker                 (continued)                                                       ...
A sequence diagram for the AVI scenarioCommunication       Atrial                     Ventricular              Heart  Gnom...
A sequence diagram for the Programming scenarioProgrammer     ReedSwitch             CoilDriver                  Communica...
2) Perform Complexity                                                   West                                              ...
West         2) Perform Complexity                                         Virginia                                       ...
2) Perform Complexity                                                          West                                       ...
West        2) Perform Complexity                                           Virginia                                      ...
West            3) Perform Severity                        Virginia                                                      U...
West            3) Perform Severity                                      Virginia                                         ...
West                                                                                                                 Virgi...
West                                                                                                                      ...
West                                                                        Virginia     4) Develop Risk Factors          ...
West          4) Develop Risk Factors                                                                                     ...
West                                                                                                                      ...
West     6) Risk Aggregation Algorithm                                                           Virginia                 ...
West            Risk Aggregation Algorithm                                                                                ...
West                                                                        Virginia          Risk Aggregation Algorithm  ...
West             Sensitivity Analysis                                                                                     ...
West                                                             Virginia                     Benefits                    ...
West                                                   Virginia       Conclusion : Benefits                      Universit...
West                                                 Virginia        Conclusion : Issues                                  ...
Questions ...
West                                                                                                                    Vi...
Upcoming SlideShare
Loading in …5
×

Architectural Level Risk Analysis for UML Dynamic Specification

475 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
475
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Architectural Level Risk Analysis for UML Dynamic Specification

  1. 1. West Virginia University Architectural-Level Risk Analysis for UML Dynamic Specifications Dr. Sherif M. Yacoub Alaa Ibrahim, and Hany H. Ammar sherif_yacoub@hp.com {ibrahim,ammar}@csee.wvu.eduHewlett-Packard Laboratories Department of Computer Science and Palo Alto, CA Electrical Engineering West Virginia University 9 th International Conference on Software Quality Management, SQM2001 18 th -20 th April, 2001 Loughborough University, Loughborough, England
  2. 2. West Virginia University Outline Research Objectives Methodology Towards an Automated Methodology Process Case Study: The Pacemaker example Conclusions
  3. 3. Automated Risk West Assessment Virginia University Research Objectives Architectural-Level Risk Assessment Methodology at the early stages of development (S. Yacoub, H. Ammar. ISSRE00, IEEE Comp. Soc., October, 2000) Automated Environment
  4. 4. Automated Risk Assessment (continued) West Virginia Architectural-Level Risk Assessment University Methodology (S. Yacoub, H. Ammar. ISSRE00, IEEE Comp. Soc., October, 2000)Utilizes: • Dynamic Metrics: Component Complexity cpx i Connector Complexity cpx ij (S. Yacoub, H. Ammar, and T. Robinson. Metrics99, November 1999) • Failure Mode Effect Analysis FMEA (MIL_STD 1629A to define Component Severity svrty i Connector Severity svrty ij) • Component Dependency Graphs CDG (adopted from: S. Yacoub, B. Cukic, and H. Ammar. ISSRE99 November 1999)Defines: • Heuristic Component Risk Factor hrf i = cpx i x svrty i • Heuristic Connector Risk Factor hrf ij = cpx ij x svrty ij • Risk Aggregation Algorithm that produces HRF appl
  5. 5. West Automated Risk Assessment Virginia Architectural-Level Risk Assessment University Methodology (continued) 6 Steps• Model the architecture of the system using simulation models (UML-RT).• Perform complexity analysis using simulation traces.• Perform severity analysis using FMEA and simulation runs.• Develop heuristic risk factors for components and connectors.• Develop Components Dependency Graph for risk assessment purposes. (System/Subsystems)• Aggregate the risk factors using the graph traversal algorithm.
  6. 6. West Automated Risk Assessment (continued) Virginia University Automated Environment Severity Analysis (Failure/Effect analysis) Severity Analyst Ranking CARA Tool Simulation Settings Inspection Viewing MacroUML Simulation Environment Simulation UML Model Log and Timing Diag. Sub Run Analysis Analysis Vi olation T able HRF Settings Violation Tool Tool Report Excel sheets Observer Rose Real Time tool MS Excel Component MS Excel Text File Processing Complexity Risk Macro Factors Macro Connector complexity Factors CDG “ hrfi and hrfij unidentified” Formatted Excel charts Violation Tables
  7. 7. Automated Risk Assessment West Virginia Automated Environment (continued) University Process Model the architecture of the system together with the risk logging capability using Rose RealTime. Adjust the simulation runs in the observer as desired. Run the simulation and get two log files containing: • Component complexities. • Component Execution Time. • A log of all the messages exchanged.
  8. 8. Automated Risk Assessment West Virginia Automated Environment University Process (continued) Process the log with Excel Risk Macro and get: • Transition Probabilities. • Connector complexities. • CDG “where Risk Factors = Severity Factors * Complexity Factors ( hrf i = cpx i x svrty i )” Perform severity analysis using FMEA and simulation runs. Traverse the CDG using the Excel traversal macro.
  9. 9. West Virginia University Example: Pacemaker Main Use Case Diagram DoctorsProgramer 1 1ProgrammingMode Programming «extend» «extend» «extend» «extend» «extend»OperationalModes 1 1 1 1 Operating_in_ AAT Operating_in_AVI Operating_in_ VVT Operating_in_ AAI 1 Operating_in_ VVI 1 1 1 1 1 PatientsHeart
  10. 10. Example: Pacemaker West Virginia University1) Develop a Simulation Model Capsule Diagram
  11. 11. West Case Study: Pacemaker (continued) Virginia Atrial statechart UniversityToOn ToOff ToOn A_Self_inhibited Idl e ToInhibited ToAVI ToTriggered A_AVI A_Self_triggered
  12. 12. West Case Study: Pacemaker (continued) Virginia Atrial statechart University T oAVI initialize Refractory A_Pace_Pulse_Done PacingV_Refract_Done_Received Time_Out Wait V_Sense_Received
  13. 13. A sequence diagram for the AVI scenarioCommunication Atrial Ventricular Heart Gnome ToON ToON ToAVI Refactoring ToAVI Refactoring RefTimeOut V Refract Done Waiting Waiting V Sense Got V Sense SensTimeOut Pacing A Pace Start Pacing A Pace Start Pace PaceTimeOut A Pace Done Refactoring Refactoring
  14. 14. A sequence diagram for the Programming scenarioProgrammer ReedSwitch CoilDriver Communication Atrial Ventricular Gnome ApplyMagnet EnableComm IDLE EnableComm IDLE ToON ToON Pulse Count = 1, SetTimer Receiving Pulse Count++, ResetTimer BitTimeout Decode(Count) Store Bit in Byte Waiting For Byte Full? Byte Yes enqueue(byte) Waiting for Bit Pulse Count =0 Receiving OR ByteTimeOut ByteTimeOut IDLE Validating IsValid? ToAVI HerezaByte(ACK) Yes Processing HerezaByte(NAK) Waiting to Send No Next Byte Waiting to ToAVI Transmit
  15. 15. 2) Perform Complexity West Virginia Analysis UniversityA Transition between Composite States in a component’s Statechart s2 init I I s1 s21 init t12 t11 t13 s11 s22 VGx(s11) + VGa(t11) + VGx(s1)+ VGa(t12) + VGe(s2) + VGa(t13) +VGe(s22) Operational Complexity of a component using the scenario profile and its complexity per scenario. |X | OCPX (oi ) = ∑ PSx × ocpxx (oi ) x =1
  16. 16. West 2) Perform Complexity Virginia University Analysis (cont’d)A) Quantify Component Complexity Factors using dynamic complexity metrics. RS CD CG AR VTProgramming ( 0.01) 8.3 67.4 24.3AVI (0.29) 53.2 46.8AAT (0.15) 100AAI (0.20) 100VVI (0.15) 100VVT (0.20) 100% of architecture complexity .083 0.674 0.243 50.428 48.572Normalized to max. complexity 0.002 0.013 0.005 1 0.963
  17. 17. 2) Perform Complexity West Virginia Analysis (cont’d) University Export Object Coupling Export Object Coupling | {Mx (oi, oj ) | oi, oj ∈ O ∧ oi ≠ oj} | EOCx(oi, oj ) = × 100 (EOC) MTx the export coupling for component Ci with respect to component Cj, is the percentage of the number of messages sent from Ci to Cj with respect to the total number of messages exchanged during the execution of the scenario x |X |EOC with scenario profiles EOC (oi, oj ) = ∑ x= 1 PSx × EOCx (oi, oj ) |X |OQFS with scenario profiles OQFS (oi ) = ∑ x= 1 PSx × OQFSx (oi )
  18. 18. West 2) Perform Complexity Virginia University Analysis (cont’d)B) Quantify Connector Complexity Factors using dynamic coupling metrics. RS CD CG AR VT Programmer HeartRS 0.0014 0.0014CD 0.003 0.011CG 0.002 0.0014 0.0014AR 0.25 1VT 0.27 0.873Programmer 0.0014 0.006Heart 0.123 0.307
  19. 19. West 3) Perform Severity Virginia University Analysis In performing severity analysis, each potential failure mode is ranked according to the consequences of that failure mode. Steps: • Identifying Failure Modes  Failure modes of individual components. (Functional faults and state-based faults)  Failure modes of individual connectors. (Interface fault analysis)
  20. 20. West 3) Perform Severity Virginia University Analysis (cont’d) Steps (cont’d): • Conducting Effect Analysis  Inject the fault.  Simulate the faulty model.  Monitor output and compare to expected output.  Identify the effect of the fault. • Rank Severity  Identify category: Minor, Marginal, Critical, or Catastrophic.  Assign severity index to each component i as (svrty i ), which takes a value of 0.25, 0.50, 0.75, and 0.95
  21. 21. West Virginia University FMEA table for the Pacemaker components Connector Name Failure Mode Cause of Failure Effect of Failure Criticality of effectsRS Failed to enable Error in translating Unable to program the Minor communication magnet command pacemaker, schedule maintenance task.CD Failed to generate Fault in developing Unable to program the Minor good command the command pacemaker, schedule maintenance task.CG Failed to validate Fault in the Cannot program the Minor command validation pacemaker, schedule procedure maintenance task. Mis-interpreting a Fault in processing Heart is continuously triggered Marginal VVT command for command routine but device is still monitored by VVI physician, need immediate fix or disable.VT No heart pluses are Heart sensor is Heart is incorrectly paced, Critical sensed though heart is malfunctioning. patient could be harmed by working fine. continuous pulses. Refract timer does not Timer not set AR and VT are in refactoring Catastrophic generate a timeout in correctly. state, no pace is generated for an AVI mode the heart, patient could die.AR Wait timer does not Timer not set AR stuck at the wait state, no Catastrophic generate a timeout in correctly. pacing is done to the heart AAI mode Worst case severity found for the RS, CD, CG, VT, and AR are Minor(0.25), Minor(0.25), Marginal(0.50), Catastrophic(0.95) and Catastrophic (0.95), respectively
  22. 22. West Virginia University FMEA table for the Pacemaker connectorsConnector Name Failure Mode Cause of Failure Effect of Failure Criticality of effectsRS-CG Failure to enable Magnet malfunctioning. Pacemaker is not programmed, Minor communication of the RS failed to generate schedule maintenance task CG message.RS-CD Unable to disable Magnet malfunctioning. Pacemaker receive bits accidentally Minor communication of the RS failed to generate from hazards but device is never CD with the correct disable message. programmed because CG is disabled, programmer schedule maintenance task.CD-Programmer Failed to acknowledge Fault in coding the Pacemaker is not programmed, Minor programming sending message schedule maintenance task.CD-CG Failed to send bytes of Inappropriate count of Pacemaker is not programmed, Minor program data to CG number of bits in a byte. schedule maintenance task.CG-AR Send incorrect Incorrect interpretation Incorrect operation mode and Marginal command (ex ToOff of program bytes incorrect rate of pacing the heart. instead of ToIdle) Device is still monitored by the physician, immediate maintenance or disable is required.CG-VT Send incorrect Incorrect interpretation Incorrect operation mode and Marginal command (ex ToOff of program bytes incorrect rate of pacing the heart. instead of ToIdle Device is still monitored by the physician, immediate maintenance or disable is required.AR-Heart Failed to sense heart in Sensor error. Heart is always paced while patient Critical AAI mode condition requires only pacing the heart when no pulse is detected Failed to pace the heart Pacing hardware device Heart could be in serious problem Catastrophic in AVI mode malfunctioning because of no pacing.VT-AR VT failed to inform Timing mismatches Failure to pace the heart. Catastrophic AR of finishing between AR and VT refractoring in AVI operation. mode
  23. 23. West Virginia 4) Develop Risk Factors University hrf i = cpx i x svrty iwhere:0 <= cpx i <= 1, is the normalized complexity level (dynamiccomplexity for components or dynamic coupling for connectors),and0<= svrty i < 1 , is the severity level for the architecture element. RS CD CG AR VT Dynamic 0.002 0.013 0.005 1 0.963 Complexity Severity 0.25 0.25 0.5 0.95 0.95 Risk Factors 0.0005 0.00325 0.0025 0.95 0.91485 Risk Factors for the components in the example
  24. 24. West 4) Develop Risk Factors Virginia University (cont’d) 1 0.9 0.8 0.7 Risk Factors 0.6 Dynamic 0.5 CBO 0.4 NAS 0.3 0.2 0.1 0 RS CD CG AR VT Comparison between risk factors based on static and dynamic metricsConnector Risk Factors RS CD CG AR VT Programmer HeartRS 0.00035 0.00035CD 0.00075 0.00275CG 0.0005 0.0007 0.0007AR 0.2375 0.95VT 0.2565 0.82935Programmer 0.00035 .0015Heart 0.11685 0.29165 Risk Factors for the connectors in the pacemaker example
  25. 25. West Virginia 5) Constructing the CDG University s <, 0, .01> <, 0, .35> <, 0, .64> t <, 0, .99> <, 0, .36> <Prog., 0,5> <, 0, .34> t <,.26,.29> <VT,0.9,40> <,3.5x10-4, .002> <AR,0.95,40> <, 0, .99> <,.24,.19> -4<,2.7x10-3,.008> <RS,5x10 ,5> <,.26,.29> <,.12,.35> <,1.5x10-3,.008> <,.29,.64> <,3.5x10-3,.005> -4 <,7x10 ,.0025> <,.95,.47> <,3.5x10-4,.005> -4 <,7x10 ,.0025> -4 <,7.5x10 ,.002> <CD, 3x10-3,5> <CG, 2.5x10-2,5> <Heart,0,5> <,5x10-4,.005> <, 0, .99> <, 0, .99> <, 0, .01> t
  26. 26. West 6) Risk Aggregation Algorithm Virginia University The algorithm expands all branches of the CDG starting from the start node. The breadth expansions of the graph represent logical "OR" paths. • translated as the summation of aggregated risk factors weighted by the transition probability along each path. The depth of each path represents the sequential execution of components: • is given by the aggregate: HRF = 1 - π i (1- hrf i )
  27. 27. West Risk Aggregation Algorithm Virginia UniversityProcedure AssessRiskParameters consumes CDG, AE appl ,(average execution time for the application) produces Risk applInitialization:R appl = R temp = 1 (temporary variables for (1-RiskFactor) )Time = 0Algorithmpush tuple <C 1 , hrf 1 , EC 1 >, Time, R tempwhile Stack not EMPTY do pop < C i , hrf i , EC i >, Time, R temp if Time > AE appl or C i = t; (terminating node) R appl += R temp ;(an OR path) else ∀ < C j ,hrf j , EC j > ∈ children(C i ) push (<C j , hrf j ,EC j >, Time += EC i , R temp = R temp *(1-hrf i )*(1-hrf ij )*PT ij ) ( AND path) endend while Risk appl = 1- R applend Procedure AssessRisk
  28. 28. West Virginia Risk Aggregation Algorithm University The algorithm can be used for • System-level Risk Assessment  The risk of the pacemaker that is found to be ~ 0.9 • Subsystem-level Risk Comparison  Complex systems are composed of many subsystems.  The algorithm can be used to obtain a risk factor for a subsystem using risk factors of its individual components.  Compare risk factors of individual subsystems. • Sensitivity Analysis  Sensitivity to Uncertainties in Component Risk Factors  Sensitivity to Uncertainties in Connector Risk Factors
  29. 29. West Sensitivity Analysis Virginia University 1.0 Overall Risk Factor of the System 0.8 R(AR) 0.6 R(VT) R(CG) 0.4 R(CD) R(RS) 0.2 0.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Risk Factor of Individual Components The Pacemaker risk factor as function of component risk factors (one at a time) 1.0 Overall System Risk Value 0.8 R(RS-CD) 0.6 R(CG-CD) R(AR-Heart) 0.4 R(VT-AR) R(VT-Heart) 0.2 0.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Risk Factor of Individual ConnectorsThe Pacemaker risk factor as function of connector risk factors (one at a time)
  30. 30. West Virginia Benefits University The approach helps in: • Deciding which components in the architecture require more development resources. • Deciding which connectors in the architecture are of highest risk. A high risk connector indicates that the interfaces between the corresponding components and the messaging protocol should be carefully designed. • Studying how uncertainties in component risk factors affect the overall risk value of the system. • Studying how uncertainties in connector risk factors affect the overall risk value of the system.
  31. 31. West Virginia Conclusion : Benefits University The methodology is applicable early at the architectural level. The methodology is based on dynamic metrics. We use dynamic metrics to account for the fact that a fault in a frequently executed component will frequently manifest itself into a failure. The methodology is based on simulation of architecture models. Simulation helps in: • Performing FMEA procedures . • Calculating the CDG parameters such as probability of transitions. • Obtaining dynamic metrics.
  32. 32. West Virginia Conclusion : Issues University Using ordinal scale for measuring severity. Effect of uncertainties in the scenario probabilities and the estimated average execution times. Scalability issues, applying the methodology to a larger case study. Methodology is limited to systems with statechart and sequence diagram specifications.
  33. 33. Questions ...
  34. 34. West Virginia Main Use Case Diagram University DoctorsProgramer 1 1ProgrammingMode Programming «extend» «extend» «extend» «extend» «extend»OperationalModes 1 1 1 1 Operating_in_ AAT Operating_in_AVI Operating_in_ VVT Operating_in_ AAI 1 Operating_in_ VVI 1 1 1 1 1 PatientsHeart

×