This document presents an automated methodology for performing architectural-level risk analysis of UML dynamic specifications. The methodology utilizes dynamic metrics to quantify component and connector complexity, failure mode effect analysis to determine severity rankings, and a risk aggregation algorithm. It is demonstrated on a case study of a pacemaker system. The methodology provides an automated way to develop risk factors for system elements based on their complexity and severity, which can help identify high-risk areas early in development.

1. West
Virginia
University
Architectural-Level Risk Analysis
for UML Dynamic Specifications
Dr. Sherif M. Yacoub Alaa Ibrahim, and Hany H. Ammar
sherif_yacoub@hp.com {ibrahim,ammar}@csee.wvu.edu
Hewlett-Packard Laboratories Department of Computer Science and
Palo Alto, CA Electrical Engineering
West Virginia University
9 th International Conference on Software Quality
Management, SQM2001
18 th -20 th April, 2001
Loughborough University, Loughborough, England

2. West
Virginia
University
Outline
 Research Objectives
 Methodology
 Towards an Automated Methodology
 Process
 Case Study: The Pacemaker example
 Conclusions

3. Automated Risk West
Assessment Virginia
University
Research Objectives
 Architectural-Level Risk Assessment
Methodology at the early stages of
development (S. Yacoub, H. Ammar. ISSRE'00, IEEE Comp. Soc., October,
2000)
 Automated Environment

4. Automated Risk Assessment (continued)
West
Virginia
Architectural-Level Risk Assessment University
Methodology
(S. Yacoub, H. Ammar. ISSRE'00, IEEE Comp. Soc., October, 2000)
Utilizes:
• Dynamic Metrics: Component Complexity cpx i Connector
Complexity cpx ij (S. Yacoub, H. Ammar, and T. Robinson. Metrics'99,
November 1999)
• Failure Mode Effect Analysis FMEA (MIL_STD 1629A to define
Component Severity svrty i Connector Severity svrty ij)
• Component Dependency Graphs CDG (adopted from: S. Yacoub, B.
Cukic, and H. Ammar. ISSRE'99 November 1999)
Defines:
• Heuristic Component Risk Factor hrf i = cpx i x svrty i
• Heuristic Connector Risk Factor hrf ij = cpx ij x svrty ij
• Risk Aggregation Algorithm that produces HRF appl

5. West
Automated Risk Assessment Virginia
Architectural-Level Risk Assessment University
Methodology (continued)
6 Steps
• Model the architecture of the system using simulation
models (UML-RT).
• Perform complexity analysis using simulation traces.
• Perform severity analysis using FMEA and simulation runs.
• Develop heuristic risk factors for components and
connectors.
• Develop Components Dependency Graph for risk
assessment purposes. (System/Subsystems)
• Aggregate the risk factors using the graph traversal
algorithm.

6. West
Automated Risk Assessment (continued)
Virginia
University
Automated Environment
Severity Analysis
(Failure/Effect analysis)
Severity
Analyst Ranking CARA Tool
Simulation
Settings
Inspection
Viewing Macro
UML Simulation Environment
Simulation
UML Model Log and Timing Diag.
Sub Run Analysis Analysis
Vi olation T able HRF
Settings Violation Tool Tool
Report Excel sheets
Observer
Rose Real Time tool MS Excel Component MS Excel
Text File Processing Complexity Risk
Macro Factors Macro
Connector
complexity
Factors
CDG “ hrfi and hrfij
unidentified”
Formatted Excel
charts
Violation Tables

7. Automated Risk Assessment West
Virginia
Automated Environment (continued)
University
Process
 Model the architecture of the system together with
the risk logging capability using Rose RealTime.
 Adjust the simulation runs in the observer as
desired.
 Run the simulation and get two log files
containing:
• Component complexities.
• Component Execution Time.
• A log of all the messages exchanged.

8. Automated Risk Assessment West
Virginia
Automated Environment University
Process (continued)
 Process the log with Excel Risk Macro and get:
• Transition Probabilities.
• Connector complexities.
• CDG “where Risk Factors = Severity Factors *
Complexity Factors ( hrf i = cpx i x svrty i )”
 Perform severity analysis using FMEA and
simulation runs.
 Traverse the CDG using the Excel traversal
macro.

9. West
Virginia
University
Example: Pacemaker
Main Use Case Diagram
DoctorsProgramer
1
1
Programming
Mode
Programming
«extend»
«extend» «extend» «extend»
«extend»
Operational
Modes 1 1 1
1 Operating_in_ AAT
Operating_in_AVI Operating_in_ VVT
Operating_in_ AAI 1 Operating_in_ VVI
1 1 1 1
1
PatientsHeart

10. Example: Pacemaker West
Virginia
University
1) Develop a Simulation Model
Capsule Diagram

11. West
Case Study: Pacemaker (continued)
Virginia
Atrial statechart University
ToOn ToOff
ToOn
A_Self_inhibited
Idl e
ToInhibited
ToAVI
ToTriggered
A_AVI A_Self_triggered

12. West
Case Study: Pacemaker (continued)
Virginia
Atrial statechart University
T oAVI
initialize Refractory
A_Pace_Pulse_Done
Pacing
V_Refract_Done_Received
Time_Out
Wait
V_Sense_Received

13. A sequence diagram for the AVI scenario
Communication Atrial Ventricular Heart
Gnome
ToON
ToON
ToAVI
Refactoring
ToAVI
Refactoring
RefTimeOut
V Refract Done
Waiting Waiting
V Sense
Got V Sense
SensTimeOut
Pacing A Pace Start
Pacing
A Pace Start
Pace
PaceTimeOut
A Pace Done
Refactoring Refactoring

14. A sequence diagram for the Programming scenario
Programmer ReedSwitch CoilDriver Communication Atrial Ventricular
Gnome
ApplyMagnet
EnableComm IDLE
EnableComm IDLE
ToON
ToON
Pulse
Count = 1, SetTimer
Receiving
Pulse
Count++,
ResetTimer
BitTimeout
Decode(Count)
Store Bit in Byte
Waiting For
Byte Full? Byte
Yes enqueue(byte)
Waiting for
Bit
Pulse
Count =0
Receiving
OR ByteTimeOut
ByteTimeOut
IDLE Validating
IsValid?
ToAVI
HerezaByte(ACK) Yes Processing
HerezaByte(NAK)
Waiting to Send
No Next Byte
Waiting to ToAVI
Transmit

15. 2) Perform Complexity West
Virginia
Analysis
University
A Transition between Composite States in a component’s Statechart
s2
init
I
I s1 s21
init
t12
t11 t13
s11 s22
VGx(s11) + VGa(t11) + VGx(s1)+ VGa(t12) + VGe(s2) + VGa(t13) +VGe(s22)
Operational Complexity of a component using the scenario
profile and its complexity per scenario.
|X |
OCPX (oi ) = ∑ PSx × ocpxx (oi )
x =1

16. West
2) Perform Complexity Virginia
University
Analysis (cont’d)
A) Quantify Component Complexity Factors
using dynamic complexity metrics.
RS CD CG AR VT
Programming ( 0.01) 8.3 67.4 24.3
AVI (0.29) 53.2 46.8
AAT (0.15) 100
AAI (0.20) 100
VVI (0.15) 100
VVT (0.20) 100
% of architecture complexity .083 0.674 0.243 50.428 48.572
Normalized to max. complexity 0.002 0.013 0.005 1 0.963

17. 2) Perform Complexity West
Virginia
Analysis (cont’d)
University
Export Object Coupling
Export Object Coupling
| {Mx (oi, oj ) | oi, oj ∈ O ∧ oi ≠ oj} |
EOCx(oi, oj ) = × 100
(EOC) MTx
the export coupling for component Ci with respect to component
Cj, is the percentage of the number of messages sent from Ci to
Cj with respect to the total number of messages exchanged
during the execution of the scenario x |X |
EOC with scenario profiles EOC (oi, oj ) = ∑
x= 1
PSx × EOCx (oi, oj )
|X |
OQFS with scenario profiles
OQFS (oi ) = ∑
x= 1
PSx × OQFSx (oi )

18. West
2) Perform Complexity Virginia
University
Analysis (cont’d)
B) Quantify Connector Complexity Factors
using dynamic coupling metrics.
RS CD CG AR VT Programmer Heart
RS 0.0014 0.0014
CD 0.003 0.011
CG 0.002 0.0014 0.0014
AR 0.25 1
VT 0.27 0.873
Programmer 0.0014 0.006
Heart 0.123 0.307

19. West
3) Perform Severity Virginia
University
Analysis
 In performing severity analysis, each potential
failure mode is ranked according to the
consequences of that failure mode.
 Steps:
• Identifying Failure Modes
 Failure modes of individual components.
(Functional faults and state-based faults)
 Failure modes of individual connectors.
(Interface fault analysis)

20. West
3) Perform Severity Virginia
University
Analysis (cont’d)
 Steps (cont’d):
• Conducting Effect Analysis
 Inject the fault.
 Simulate the faulty model.
 Monitor output and compare to expected output.
 Identify the effect of the fault.
• Rank Severity
 Identify category: Minor, Marginal, Critical, or
Catastrophic.
 Assign severity index to each component i as (svrty i ),
which takes a value of 0.25, 0.50, 0.75, and 0.95

21. West
Virginia
University
FMEA table for the Pacemaker components
Connector Name Failure Mode Cause of Failure Effect of Failure Criticality of effects
RS Failed to enable Error in translating Unable to program the Minor
communication magnet command pacemaker, schedule
maintenance task.
CD Failed to generate Fault in developing Unable to program the Minor
good command the command pacemaker, schedule
maintenance task.
CG Failed to validate Fault in the Cannot program the Minor
command validation pacemaker, schedule
procedure maintenance task.
Mis-interpreting a Fault in processing Heart is continuously triggered Marginal
VVT command for command routine but device is still monitored by
VVI physician, need immediate fix
or disable.
VT No heart pluses are Heart sensor is Heart is incorrectly paced, Critical
sensed though heart is malfunctioning. patient could be harmed by
working fine. continuous pulses.
Refract timer does not Timer not set AR and VT are in refactoring Catastrophic
generate a timeout in correctly. state, no pace is generated for
an AVI mode the heart, patient could die.
AR Wait timer does not Timer not set AR stuck at the wait state, no Catastrophic
generate a timeout in correctly. pacing is done to the heart
AAI mode
 Worst case severity found for the RS, CD, CG, VT, and AR
are Minor(0.25), Minor(0.25), Marginal(0.50),
Catastrophic(0.95) and Catastrophic (0.95), respectively

22. West
Virginia
University
FMEA table for the Pacemaker connectors
Connector Name Failure Mode Cause of Failure Effect of Failure Criticality of effects
RS-CG Failure to enable Magnet malfunctioning. Pacemaker is not programmed, Minor
communication of the RS failed to generate schedule maintenance task
CG message.
RS-CD Unable to disable Magnet malfunctioning. Pacemaker receive bits accidentally Minor
communication of the RS failed to generate from hazards but device is never
CD with the correct disable message. programmed because CG is disabled,
programmer schedule maintenance task.
CD-Programmer Failed to acknowledge Fault in coding the Pacemaker is not programmed, Minor
programming sending message schedule maintenance task.
CD-CG Failed to send bytes of Inappropriate count of Pacemaker is not programmed, Minor
program data to CG number of bits in a byte. schedule maintenance task.
CG-AR Send incorrect Incorrect interpretation Incorrect operation mode and Marginal
command (ex ToOff of program bytes incorrect rate of pacing the heart.
instead of ToIdle) Device is still monitored by the
physician, immediate maintenance or
disable is required.
CG-VT Send incorrect Incorrect interpretation Incorrect operation mode and Marginal
command (ex ToOff of program bytes incorrect rate of pacing the heart.
instead of ToIdle Device is still monitored by the
physician, immediate maintenance or
disable is required.
AR-Heart Failed to sense heart in Sensor error. Heart is always paced while patient Critical
AAI mode condition requires only pacing the
heart when no pulse is detected
Failed to pace the heart Pacing hardware device Heart could be in serious problem Catastrophic
in AVI mode malfunctioning because of no pacing.
VT-AR VT failed to inform Timing mismatches Failure to pace the heart. Catastrophic
AR of finishing between AR and VT
refractoring in AVI operation.
mode

23. West
Virginia
4) Develop Risk Factors
University
hrf i = cpx i x svrty i
where:
0 <= cpx i <= 1, is the normalized complexity level (dynamic
complexity for components or dynamic coupling for connectors),
and
0<= svrty i < 1 , is the severity level for the architecture element.
RS CD CG AR VT
Dynamic 0.002 0.013 0.005 1 0.963
Complexity
Severity 0.25 0.25 0.5 0.95 0.95
Risk Factors 0.0005 0.00325 0.0025 0.95 0.91485
Risk Factors for the components in the example

24. West
4) Develop Risk Factors
Virginia
University
(cont’d)
1
0.9
0.8
0.7
Risk Factors
0.6 Dynamic
0.5 CBO
0.4 NAS
0.3
0.2
0.1
0
RS CD CG AR VT
Comparison between risk factors based on static and dynamic metrics
Connector Risk Factors RS CD CG AR VT Programmer Heart
RS 0.00035 0.00035
CD 0.00075 0.00275
CG 0.0005 0.0007 0.0007
AR 0.2375 0.95
VT 0.2565 0.82935
Programmer 0.00035 .0015
Heart 0.11685 0.29165
Risk Factors for the connectors in the pacemaker example

25. West
Virginia
5) Constructing the CDG
University
s
<, 0, .01> <, 0, .35>
<, 0, .64> t
<, 0, .99> <, 0, .36>
<Prog., 0,5> <, 0, .34>
t
<,.26,.29> <VT,0.9,40>
<,3.5x10-4, .002> <AR,0.95,40>
<, 0, .99>
<,.24,.19>
-4
<,2.7x10-3,.008>
<RS,5x10 ,5> <,.26,.29>
<,.12,.35>
<,1.5x10-3,.008> <,.29,.64>
<,3.5x10-3,.005> -4
<,7x10 ,.0025>
<,.95,.47>
<,3.5x10-4,.005>
-4
<,7x10 ,.0025>
-4
<,7.5x10 ,.002>
<CD, 3x10-3,5> <CG, 2.5x10-2,5>
<Heart,0,5>
<,5x10-4,.005>
<, 0, .99>
<, 0, .99> <, 0, .01>
t

26. West
6) Risk Aggregation Algorithm
Virginia
University
 The algorithm expands all branches of the CDG starting
from the start node.
 The breadth expansions of the graph represent logical
"OR" paths.
• translated as the summation of
aggregated risk factors weighted by the
transition probability along each path.
 The depth of each path represents the sequential
execution of components:
• is given by the aggregate: HRF = 1 - π i (1-
hrf i )

27. West
Risk Aggregation Algorithm
Virginia
University
Procedure AssessRisk
Parameters
consumes CDG, AE appl ,(average execution time for the application)
produces Risk appl
Initialization:
R appl = R temp = 1 (temporary variables for (1-RiskFactor) )
Time = 0
Algorithm
push tuple <C 1 , hrf 1 , EC 1 >, Time, R temp
while Stack not EMPTY do
pop < C i , hrf i , EC i >, Time, R temp
if Time > AE appl or C i = t; (terminating node)
R appl += R temp ;(an OR path)
else
∀ < C j ,hrf j , EC j > ∈ children(C i )
push (<C j , hrf j ,EC j >, Time += EC i ,
R temp = R temp *(1-hrf i )*(1-hrf ij )*PT ij ) ( AND path)
end
end while
Risk appl = 1- R appl
end Procedure AssessRisk

28. West
Virginia
Risk Aggregation Algorithm
University
 The algorithm can be used for
• System-level Risk Assessment
 The risk of the pacemaker that is found to be ~ 0.9
• Subsystem-level Risk Comparison
 Complex systems are composed of many subsystems.
 The algorithm can be used to obtain a risk factor for a subsystem
using risk factors of its individual components.
 Compare risk factors of individual subsystems.
• Sensitivity Analysis
 Sensitivity to Uncertainties in Component Risk Factors
 Sensitivity to Uncertainties in Connector Risk Factors

29. West
Sensitivity Analysis
Virginia
University
1.0
Overall Risk Factor of the System
0.8
R(AR)
0.6 R(VT)
R(CG)
0.4 R(CD)
R(RS)
0.2
0.0
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Risk Factor of Individual Components
The Pacemaker risk factor as function of component risk factors (one at a time)
1.0
Overall System Risk Value
0.8
R(RS-CD)
0.6 R(CG-CD)
R(AR-Heart)
0.4 R(VT-AR)
R(VT-Heart)
0.2
0.0
0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Risk Factor of Individual Connectors
The Pacemaker risk factor as function of connector risk factors (one at a time)

30. West
Virginia
Benefits University
 The approach helps in:
• Deciding which components in the architecture require
more development resources.
• Deciding which connectors in the architecture are of
highest risk. A high risk connector indicates that the
interfaces between the corresponding components and
the messaging protocol should be carefully designed.
• Studying how uncertainties in component risk factors
affect the overall risk value of the system.
• Studying how uncertainties in connector risk factors
affect the overall risk value of the system.

31. West
Virginia
Conclusion : Benefits University
 The methodology is applicable early at the
architectural level.
 The methodology is based on dynamic metrics.
We use dynamic metrics to account for the fact
that a fault in a frequently executed component
will frequently manifest itself into a failure.
 The methodology is based on simulation of
architecture models. Simulation helps in:
• Performing FMEA procedures .
• Calculating the CDG parameters such as probability
of transitions.
• Obtaining dynamic metrics.

32. West
Virginia
Conclusion : Issues
University
 Using ordinal scale for measuring severity.
 Effect of uncertainties in the scenario
probabilities and the estimated average execution
times.
 Scalability issues, applying the methodology to a
larger case study.
 Methodology is limited to systems with statechart
and sequence diagram specifications.

33. Questions ...

34. West
Virginia
Main Use Case Diagram University
DoctorsProgramer
1
1
Programming
Mode
Programming
«extend»
«extend» «extend» «extend»
«extend»
Operational
Modes 1 1 1
1 Operating_in_ AAT
Operating_in_AVI Operating_in_ VVT
Operating_in_ AAI 1 Operating_in_ VVI
1 1 1 1
1
PatientsHeart