Architectural Level Risk Analysis for UML Dynamic Specification
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Architectural Level Risk Analysis for UML Dynamic Specification

  • 522 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
522
On Slideshare
522
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. West Virginia University Architectural-Level Risk Analysis for UML Dynamic Specifications Dr. Sherif M. Yacoub Alaa Ibrahim, and Hany H. Ammar sherif_yacoub@hp.com {ibrahim,ammar}@csee.wvu.eduHewlett-Packard Laboratories Department of Computer Science and Palo Alto, CA Electrical Engineering West Virginia University 9 th International Conference on Software Quality Management, SQM2001 18 th -20 th April, 2001 Loughborough University, Loughborough, England
  • 2. West Virginia University Outline Research Objectives Methodology Towards an Automated Methodology Process Case Study: The Pacemaker example Conclusions
  • 3. Automated Risk West Assessment Virginia University Research Objectives Architectural-Level Risk Assessment Methodology at the early stages of development (S. Yacoub, H. Ammar. ISSRE00, IEEE Comp. Soc., October, 2000) Automated Environment
  • 4. Automated Risk Assessment (continued) West Virginia Architectural-Level Risk Assessment University Methodology (S. Yacoub, H. Ammar. ISSRE00, IEEE Comp. Soc., October, 2000)Utilizes: • Dynamic Metrics: Component Complexity cpx i Connector Complexity cpx ij (S. Yacoub, H. Ammar, and T. Robinson. Metrics99, November 1999) • Failure Mode Effect Analysis FMEA (MIL_STD 1629A to define Component Severity svrty i Connector Severity svrty ij) • Component Dependency Graphs CDG (adopted from: S. Yacoub, B. Cukic, and H. Ammar. ISSRE99 November 1999)Defines: • Heuristic Component Risk Factor hrf i = cpx i x svrty i • Heuristic Connector Risk Factor hrf ij = cpx ij x svrty ij • Risk Aggregation Algorithm that produces HRF appl
  • 5. West Automated Risk Assessment Virginia Architectural-Level Risk Assessment University Methodology (continued) 6 Steps• Model the architecture of the system using simulation models (UML-RT).• Perform complexity analysis using simulation traces.• Perform severity analysis using FMEA and simulation runs.• Develop heuristic risk factors for components and connectors.• Develop Components Dependency Graph for risk assessment purposes. (System/Subsystems)• Aggregate the risk factors using the graph traversal algorithm.
  • 6. West Automated Risk Assessment (continued) Virginia University Automated Environment Severity Analysis (Failure/Effect analysis) Severity Analyst Ranking CARA Tool Simulation Settings Inspection Viewing MacroUML Simulation Environment Simulation UML Model Log and Timing Diag. Sub Run Analysis Analysis Vi olation T able HRF Settings Violation Tool Tool Report Excel sheets Observer Rose Real Time tool MS Excel Component MS Excel Text File Processing Complexity Risk Macro Factors Macro Connector complexity Factors CDG “ hrfi and hrfij unidentified” Formatted Excel charts Violation Tables
  • 7. Automated Risk Assessment West Virginia Automated Environment (continued) University Process Model the architecture of the system together with the risk logging capability using Rose RealTime. Adjust the simulation runs in the observer as desired. Run the simulation and get two log files containing: • Component complexities. • Component Execution Time. • A log of all the messages exchanged.
  • 8. Automated Risk Assessment West Virginia Automated Environment University Process (continued) Process the log with Excel Risk Macro and get: • Transition Probabilities. • Connector complexities. • CDG “where Risk Factors = Severity Factors * Complexity Factors ( hrf i = cpx i x svrty i )” Perform severity analysis using FMEA and simulation runs. Traverse the CDG using the Excel traversal macro.
  • 9. West Virginia University Example: Pacemaker Main Use Case Diagram DoctorsProgramer 1 1ProgrammingMode Programming «extend» «extend» «extend» «extend» «extend»OperationalModes 1 1 1 1 Operating_in_ AAT Operating_in_AVI Operating_in_ VVT Operating_in_ AAI 1 Operating_in_ VVI 1 1 1 1 1 PatientsHeart
  • 10. Example: Pacemaker West Virginia University1) Develop a Simulation Model Capsule Diagram
  • 11. West Case Study: Pacemaker (continued) Virginia Atrial statechart UniversityToOn ToOff ToOn A_Self_inhibited Idl e ToInhibited ToAVI ToTriggered A_AVI A_Self_triggered
  • 12. West Case Study: Pacemaker (continued) Virginia Atrial statechart University T oAVI initialize Refractory A_Pace_Pulse_Done PacingV_Refract_Done_Received Time_Out Wait V_Sense_Received
  • 13. A sequence diagram for the AVI scenarioCommunication Atrial Ventricular Heart Gnome ToON ToON ToAVI Refactoring ToAVI Refactoring RefTimeOut V Refract Done Waiting Waiting V Sense Got V Sense SensTimeOut Pacing A Pace Start Pacing A Pace Start Pace PaceTimeOut A Pace Done Refactoring Refactoring
  • 14. A sequence diagram for the Programming scenarioProgrammer ReedSwitch CoilDriver Communication Atrial Ventricular Gnome ApplyMagnet EnableComm IDLE EnableComm IDLE ToON ToON Pulse Count = 1, SetTimer Receiving Pulse Count++, ResetTimer BitTimeout Decode(Count) Store Bit in Byte Waiting For Byte Full? Byte Yes enqueue(byte) Waiting for Bit Pulse Count =0 Receiving OR ByteTimeOut ByteTimeOut IDLE Validating IsValid? ToAVI HerezaByte(ACK) Yes Processing HerezaByte(NAK) Waiting to Send No Next Byte Waiting to ToAVI Transmit
  • 15. 2) Perform Complexity West Virginia Analysis UniversityA Transition between Composite States in a component’s Statechart s2 init I I s1 s21 init t12 t11 t13 s11 s22 VGx(s11) + VGa(t11) + VGx(s1)+ VGa(t12) + VGe(s2) + VGa(t13) +VGe(s22) Operational Complexity of a component using the scenario profile and its complexity per scenario. |X | OCPX (oi ) = ∑ PSx × ocpxx (oi ) x =1
  • 16. West 2) Perform Complexity Virginia University Analysis (cont’d)A) Quantify Component Complexity Factors using dynamic complexity metrics. RS CD CG AR VTProgramming ( 0.01) 8.3 67.4 24.3AVI (0.29) 53.2 46.8AAT (0.15) 100AAI (0.20) 100VVI (0.15) 100VVT (0.20) 100% of architecture complexity .083 0.674 0.243 50.428 48.572Normalized to max. complexity 0.002 0.013 0.005 1 0.963
  • 17. 2) Perform Complexity West Virginia Analysis (cont’d) University Export Object Coupling Export Object Coupling | {Mx (oi, oj ) | oi, oj ∈ O ∧ oi ≠ oj} | EOCx(oi, oj ) = × 100 (EOC) MTx the export coupling for component Ci with respect to component Cj, is the percentage of the number of messages sent from Ci to Cj with respect to the total number of messages exchanged during the execution of the scenario x |X |EOC with scenario profiles EOC (oi, oj ) = ∑ x= 1 PSx × EOCx (oi, oj ) |X |OQFS with scenario profiles OQFS (oi ) = ∑ x= 1 PSx × OQFSx (oi )
  • 18. West 2) Perform Complexity Virginia University Analysis (cont’d)B) Quantify Connector Complexity Factors using dynamic coupling metrics. RS CD CG AR VT Programmer HeartRS 0.0014 0.0014CD 0.003 0.011CG 0.002 0.0014 0.0014AR 0.25 1VT 0.27 0.873Programmer 0.0014 0.006Heart 0.123 0.307
  • 19. West 3) Perform Severity Virginia University Analysis In performing severity analysis, each potential failure mode is ranked according to the consequences of that failure mode. Steps: • Identifying Failure Modes  Failure modes of individual components. (Functional faults and state-based faults)  Failure modes of individual connectors. (Interface fault analysis)
  • 20. West 3) Perform Severity Virginia University Analysis (cont’d) Steps (cont’d): • Conducting Effect Analysis  Inject the fault.  Simulate the faulty model.  Monitor output and compare to expected output.  Identify the effect of the fault. • Rank Severity  Identify category: Minor, Marginal, Critical, or Catastrophic.  Assign severity index to each component i as (svrty i ), which takes a value of 0.25, 0.50, 0.75, and 0.95
  • 21. West Virginia University FMEA table for the Pacemaker components Connector Name Failure Mode Cause of Failure Effect of Failure Criticality of effectsRS Failed to enable Error in translating Unable to program the Minor communication magnet command pacemaker, schedule maintenance task.CD Failed to generate Fault in developing Unable to program the Minor good command the command pacemaker, schedule maintenance task.CG Failed to validate Fault in the Cannot program the Minor command validation pacemaker, schedule procedure maintenance task. Mis-interpreting a Fault in processing Heart is continuously triggered Marginal VVT command for command routine but device is still monitored by VVI physician, need immediate fix or disable.VT No heart pluses are Heart sensor is Heart is incorrectly paced, Critical sensed though heart is malfunctioning. patient could be harmed by working fine. continuous pulses. Refract timer does not Timer not set AR and VT are in refactoring Catastrophic generate a timeout in correctly. state, no pace is generated for an AVI mode the heart, patient could die.AR Wait timer does not Timer not set AR stuck at the wait state, no Catastrophic generate a timeout in correctly. pacing is done to the heart AAI mode Worst case severity found for the RS, CD, CG, VT, and AR are Minor(0.25), Minor(0.25), Marginal(0.50), Catastrophic(0.95) and Catastrophic (0.95), respectively
  • 22. West Virginia University FMEA table for the Pacemaker connectorsConnector Name Failure Mode Cause of Failure Effect of Failure Criticality of effectsRS-CG Failure to enable Magnet malfunctioning. Pacemaker is not programmed, Minor communication of the RS failed to generate schedule maintenance task CG message.RS-CD Unable to disable Magnet malfunctioning. Pacemaker receive bits accidentally Minor communication of the RS failed to generate from hazards but device is never CD with the correct disable message. programmed because CG is disabled, programmer schedule maintenance task.CD-Programmer Failed to acknowledge Fault in coding the Pacemaker is not programmed, Minor programming sending message schedule maintenance task.CD-CG Failed to send bytes of Inappropriate count of Pacemaker is not programmed, Minor program data to CG number of bits in a byte. schedule maintenance task.CG-AR Send incorrect Incorrect interpretation Incorrect operation mode and Marginal command (ex ToOff of program bytes incorrect rate of pacing the heart. instead of ToIdle) Device is still monitored by the physician, immediate maintenance or disable is required.CG-VT Send incorrect Incorrect interpretation Incorrect operation mode and Marginal command (ex ToOff of program bytes incorrect rate of pacing the heart. instead of ToIdle Device is still monitored by the physician, immediate maintenance or disable is required.AR-Heart Failed to sense heart in Sensor error. Heart is always paced while patient Critical AAI mode condition requires only pacing the heart when no pulse is detected Failed to pace the heart Pacing hardware device Heart could be in serious problem Catastrophic in AVI mode malfunctioning because of no pacing.VT-AR VT failed to inform Timing mismatches Failure to pace the heart. Catastrophic AR of finishing between AR and VT refractoring in AVI operation. mode
  • 23. West Virginia 4) Develop Risk Factors University hrf i = cpx i x svrty iwhere:0 <= cpx i <= 1, is the normalized complexity level (dynamiccomplexity for components or dynamic coupling for connectors),and0<= svrty i < 1 , is the severity level for the architecture element. RS CD CG AR VT Dynamic 0.002 0.013 0.005 1 0.963 Complexity Severity 0.25 0.25 0.5 0.95 0.95 Risk Factors 0.0005 0.00325 0.0025 0.95 0.91485 Risk Factors for the components in the example
  • 24. West 4) Develop Risk Factors Virginia University (cont’d) 1 0.9 0.8 0.7 Risk Factors 0.6 Dynamic 0.5 CBO 0.4 NAS 0.3 0.2 0.1 0 RS CD CG AR VT Comparison between risk factors based on static and dynamic metricsConnector Risk Factors RS CD CG AR VT Programmer HeartRS 0.00035 0.00035CD 0.00075 0.00275CG 0.0005 0.0007 0.0007AR 0.2375 0.95VT 0.2565 0.82935Programmer 0.00035 .0015Heart 0.11685 0.29165 Risk Factors for the connectors in the pacemaker example
  • 25. West Virginia 5) Constructing the CDG University s <, 0, .01> <, 0, .35> <, 0, .64> t <, 0, .99> <, 0, .36> <Prog., 0,5> <, 0, .34> t <,.26,.29> <VT,0.9,40> <,3.5x10-4, .002> <AR,0.95,40> <, 0, .99> <,.24,.19> -4<,2.7x10-3,.008> <RS,5x10 ,5> <,.26,.29> <,.12,.35> <,1.5x10-3,.008> <,.29,.64> <,3.5x10-3,.005> -4 <,7x10 ,.0025> <,.95,.47> <,3.5x10-4,.005> -4 <,7x10 ,.0025> -4 <,7.5x10 ,.002> <CD, 3x10-3,5> <CG, 2.5x10-2,5> <Heart,0,5> <,5x10-4,.005> <, 0, .99> <, 0, .99> <, 0, .01> t
  • 26. West 6) Risk Aggregation Algorithm Virginia University The algorithm expands all branches of the CDG starting from the start node. The breadth expansions of the graph represent logical "OR" paths. • translated as the summation of aggregated risk factors weighted by the transition probability along each path. The depth of each path represents the sequential execution of components: • is given by the aggregate: HRF = 1 - π i (1- hrf i )
  • 27. West Risk Aggregation Algorithm Virginia UniversityProcedure AssessRiskParameters consumes CDG, AE appl ,(average execution time for the application) produces Risk applInitialization:R appl = R temp = 1 (temporary variables for (1-RiskFactor) )Time = 0Algorithmpush tuple <C 1 , hrf 1 , EC 1 >, Time, R tempwhile Stack not EMPTY do pop < C i , hrf i , EC i >, Time, R temp if Time > AE appl or C i = t; (terminating node) R appl += R temp ;(an OR path) else ∀ < C j ,hrf j , EC j > ∈ children(C i ) push (<C j , hrf j ,EC j >, Time += EC i , R temp = R temp *(1-hrf i )*(1-hrf ij )*PT ij ) ( AND path) endend while Risk appl = 1- R applend Procedure AssessRisk
  • 28. West Virginia Risk Aggregation Algorithm University The algorithm can be used for • System-level Risk Assessment  The risk of the pacemaker that is found to be ~ 0.9 • Subsystem-level Risk Comparison  Complex systems are composed of many subsystems.  The algorithm can be used to obtain a risk factor for a subsystem using risk factors of its individual components.  Compare risk factors of individual subsystems. • Sensitivity Analysis  Sensitivity to Uncertainties in Component Risk Factors  Sensitivity to Uncertainties in Connector Risk Factors
  • 29. West Sensitivity Analysis Virginia University 1.0 Overall Risk Factor of the System 0.8 R(AR) 0.6 R(VT) R(CG) 0.4 R(CD) R(RS) 0.2 0.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Risk Factor of Individual Components The Pacemaker risk factor as function of component risk factors (one at a time) 1.0 Overall System Risk Value 0.8 R(RS-CD) 0.6 R(CG-CD) R(AR-Heart) 0.4 R(VT-AR) R(VT-Heart) 0.2 0.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Risk Factor of Individual ConnectorsThe Pacemaker risk factor as function of connector risk factors (one at a time)
  • 30. West Virginia Benefits University The approach helps in: • Deciding which components in the architecture require more development resources. • Deciding which connectors in the architecture are of highest risk. A high risk connector indicates that the interfaces between the corresponding components and the messaging protocol should be carefully designed. • Studying how uncertainties in component risk factors affect the overall risk value of the system. • Studying how uncertainties in connector risk factors affect the overall risk value of the system.
  • 31. West Virginia Conclusion : Benefits University The methodology is applicable early at the architectural level. The methodology is based on dynamic metrics. We use dynamic metrics to account for the fact that a fault in a frequently executed component will frequently manifest itself into a failure. The methodology is based on simulation of architecture models. Simulation helps in: • Performing FMEA procedures . • Calculating the CDG parameters such as probability of transitions. • Obtaining dynamic metrics.
  • 32. West Virginia Conclusion : Issues University Using ordinal scale for measuring severity. Effect of uncertainties in the scenario probabilities and the estimated average execution times. Scalability issues, applying the methodology to a larger case study. Methodology is limited to systems with statechart and sequence diagram specifications.
  • 33. Questions ...
  • 34. West Virginia Main Use Case Diagram University DoctorsProgramer 1 1ProgrammingMode Programming «extend» «extend» «extend» «extend» «extend»OperationalModes 1 1 1 1 Operating_in_ AAT Operating_in_AVI Operating_in_ VVT Operating_in_ AAI 1 Operating_in_ VVI 1 1 1 1 1 PatientsHeart