Software Reliability and Safety

1. Software Reliability, Safety and
Testing for Industrial Applications
Dr. Ajeet Kumar

2. ➔ Introduction: The Need of Reliable and Safe Software
➔ Challenges and Issue
➔ System Reliability and Failure Distribution
➔ Hardware Reliability Vs Software Reliability
➔ Software Reliability- Reliability Prediction Models
➔ Software Safety- Functional Safety
➔ Summary
Agenda

3. ▪ Ph.D. (Reliability Engineering): IIT Kharagpur, India.
▪ M.Tech (Software Engineering): MNNIT, Allahabad, India.
Education Current Role & Credentials
Project Contribution and Earlier Affiliations
System Assurance &
RAMS Expert
India
Malaysia
Indonesia
Germany
France
UK
USA
Japan
Qatar
Australia
UAE
HS2 Phase-1 London (UK)
GFR Melbourne (Australia)
MRTS/RRTS (India)
Agra-Kanpur (India)
R151 (Singapore)
Etihad Rail (Abu Dhabi)
Dubai Metro (Dubai)
SWR (UK)
Expo link 2020 (Dubai)
DFCC (India)
LTR 3 (Malaysia)
Jakarta LRT (Indonesia)
Network Rail (UK)
Jakarta MRT (Indonesia)
Delhi Metro Line-8 (India)
Doha Metro (Qatar)
Sydney Metro (Australia)
Hyderabad Metro (India)
Invensys Rail (UK/USA)
About me
@1000 hrs.
Invited Lectures
@30
Research
Articles
@200
Citation Index
Book on
Software Reliability

4. ▪ 2015: Survey of Algorithms on Maximum Clique Problem,
International Journal, India
▪ 2014: Opinion Mining & Sentiment Analysis for Social Media using
Fuzzy Logic, International conference, India.
▪ 2014: RAMS Management for a Complex Railway System: A
Case Study, International Symposium, India.
▪ 2013: Safety Analysis of Automatic Door Operation for Metro
Train: A Case, Springer, International conference, India.
▪ 2012: Cost Effective Reliability Centric Validation Model for
Automotive ECUs, IEEE Explore.
▪ 2012: Successive Software Reliability Growth Model: A Practical
Approach, International Symposium, India.
▪ 2012: A Fuzzy Model for Early Software Quality Prediction and
Module Ranking, International Journal, India
▪ 2011: Early fault detection model using integrated and cost-
effective test case prioritization, International Journal, India.
▪ 2010: Test Effort Optimization by Prediction and Ranking of Fault-
prone Software Module, IEEE Xplore.
▪ 2010: Fault Prediction Model by Fuzzy Profile Development of
Reliability Relevant Software Metrics, International Journal, USA.
▪ 2010: Predicting Fault-prone Software Module Using Data Mining
Technique and Fuzzy Logic, International Journal, India.
▪ 2010: Modified BUSTRAP: An Optimal BUS TRAvel Planner for
Commuters using Mobile”, International Journal, India.
▪ 2010: Multistage Fault Prediction Model Using Process Level
Software Metrics”, DQM Research Center, Serbia.
▪ 2010: An Early Software Fault Prediction Model using Process
Maturity and Software Metrics, International Journal, India
▪ 2009: A Fuzzy Model for Early Software Fault Prediction using
Process Maturity & Software Metrics, International Journal, India.
▪ 2007: Digitally Signed SMS for Business Transaction
Research Contributions
Research
papers,
articles
and
patent
▪ 2021: Patent: An Artificial Intelligence Based Human
Anomaly Detection System in Public Surveillance
Videos
▪ 2021: Universally domain adaptive algorithm for
sentiment classification using transfer learning approach
▪ 2020: Domain Adaptive Model For Sentiment
Classification Using Deep Learning Approach, India.
▪ 2019: Efficient Topic Level Opinion Mining and
Sentiment Analysis Algorithm using Latent Dirichlet
Allocation Model,
▪ 2019: Efficient Multilevel Polarity Sentiment
Classification Algorithm using Support Vector Machine
and Fuzzy Logic.
▪ 2018: System Assurance, Railway RAMS and
Applicable Standards, Innorail 2018, Lucknow, India
▪ 2018: Pandey A. K., B. Vamshi Krishna, & A. P. Siva
Kumar, Feature based opinion Mining & Sentiment
Analysis using fuzzy logic, Springer Briefs.
▪ 2018: Pandey A. K., B. Vamshi Krishna, & A. P. Siva
Kumar, Topic Model Based Opinion Mining and
Sentiment Analysis, IEEE Conference, India.
▪ 2017: Reliability, Availability, Maintainability, Safety
(RAMS) and EMI/EMC for Railway Electrical System;
Proceedings of Institution of Engineers, Malaysia.
▪ 2017: Systems Assurance for Urban Railway Operation,
Seminar on Railway Electrical Systems for LRT/MRT,
Proceedings of Institution of Engineers, Malaysia
▪ 2016: Progressive RAMS Assurance & Management for
Railway Projects, Rail Reliability and Safety Conference,
Mumbai, India
▪ 2015: RAM Apportionment Model for Mass Rapid
Transit Systems, IEEE Explore.
▪ 2015: Software Safety Assurance for Metro Railways,
Business Magazine, Traffic Infra Tech, India.

5. Dr. Ajeet Kumar
Need for the Safe and Reliable Software
◆ Software are widely used to control and monitor many applications across the
industries.
◆ Software failures in safety-critical systems can have disastrous consequences such
as loss of financials assets, loss of mission and loss of human lives.
 Railways: Signalling failure, Level crossing failure, etc;
 Automotive:
 Avionics:
 Medical Devices:
◆ Demand for higher standards in software quality and reliability is required as
 Customer awareness, global market, regulation, legal requirements, etc..
 Cost of failure; cost of maintenance, downtime, etc.

6. Dr. Ajeet Kumar
Software Reliability: Challenges & Issues
• The challenges:
• Applicability of software keeps on increasing from basic home appliances to safety critical
applications. System Failures due to the software failure are very common now a days.
• Size, complexity and dependency on software based systems are growing.
• Software reliability becomes a challenging objective for both developer as well as user.
• Developer: How to develop fault free software (system)?
• User: How to select a reliable (fault-free) System?
• The issues:
• Software does not wear out (independent of time): how to apply fundamental reliability
principles and method can be applied?
• Only Systematic Failure no Random Failure……..
• No standards failure data (library), getting software failure data……..
• Infinite Failure Mode Type…….
• Testing Challenges: can not be tested exhaustively…..

7. Dr. Ajeet Kumar
System Reliability and Failure Distribution
• System Reliability:
• Hardware Reliability
• Software Reliability
• Human Reliability
A sample of project specific data
Failure cause distribution for Electronic Systems

8. Dr. Ajeet Kumar
Error, Fault (Defect) and Failure
Error
Fault/Defect
Failure
Can lead to
Can lead to
May be
Systematic and/or Random
Systematic and/or Random
Systematic and/or Random
May be
May be
Many Railway accidents, automotive recalls, medical device malfunction are growing list of software failure. In
many cases, these failure are due to the software failure which are the part of system/functions.

9. Dr. Ajeet Kumar
Failure Rate and Bath-tub Curve: Hardware Vs. Software
Hardware Bathtub Curve
Software Bathtub Curve

10. Dr. Ajeet Kumar
Failure Classification
Failure may be:
• Transient: Occurs only with
certain inputs.
• Permanent: Occurs with all
inputs.
• Recoverable: System can recover
without operator intervention.
• Unrecoverable: Operator
intervention needed to recover
from failure.
• Non-corrupting: Failure does not
corrupt systems.
• Corrupting: Failure corrupts
system state or data.
Fault:
• Defect in system implementationthat
causes the failure when executed.
• Developer oriented view.
Failure:
• Deviation of system behavior when
executed.
• It is user oriented view.

11. Dr. Ajeet Kumar
Hardware Software
Experience wear out. Does not wear out.
Failures are usually preceded by warning and can be
anticipated.
Failures are rarely preceded by warning and can’t be
anticipated.
Failure rate can be DFR/CFR/IFR i.e. bathtub curve. Failure rate does not increase.
Failures are caused by material deterioration, random
failures, design errors, misuse and environment.
Failures are caused by incorrect logic, incorrect statement or
input data.
Hardware repairs restore the original condition. Software repairs establish a new piece of software.
Can be tested exhaustively. Can not be tested exhaustively .
Failures occurs even if they are not used due to material
deterioration.
Failures never occurs if they are not used.
Components can be standardized. Software components can’t be standardized.
Hardware failures are systematic and random. Software failures are always systematic failures
Reliability: Hardware Vs. Software
Note: Software Safety, Software Reliability, and Software Security are different in with respect to the scope & objective.

12. Dr. Ajeet Kumar
Software Reliability & Safety: Systematic Failure
Software reliability: probability that a
software system or component to perform
its intended function under the specified
operating conditions over the specified
period of time i.e. failure-free operation.
A software failure is defined as “the
deviation of the program behavior from
requirements,”. Software failure occurs
when faults got executed. Faults are the
error or defects in the software.
Software faults are the root making
software unreliable.
Faults may arise across the various
stages of software development
yielding requirements faults, design
faults, and coding faults.
Insist on systematic
process to develop
a fault free software
Error Fault Failure

13. Generic Software Development Processes
Requirement
Analysis
Initial
Design
Detailed
Design
Final
Design
Coding/
Implementation
Verification and
Testing Install, Use and
Maintenance
CONCEPT
Definition Phase Development Phase Maintenance Phase
Technical & Commercial Regression Testing
Development Testing
As per the Standard Requirements for specific Industry such as Automotive, Railways, Medical Devices, etc.

14. SOFTWARE RELIABILITY

15. Dr. Ajeet Kumar
Software Reliability
• Software Reliability is defined as:
• The probability of failure-free software operation for a specified period of time
(mission time) in a specified environment
• Definition compatible with hardware reliability (failure mechanism differs)
• Cannot be defined objectively.
• Software system is very complex.
• Complex human nature is involved in the development of software.
• Requires operational profile for its definition.
• The operational profile defines the expected pattern of software usage.
• Must consider fault consequences ( for safety applications)
• Not all faults are equally serious.
• System is perceived as more unreliable if there are more serious faults.

16. Dr. Ajeet Kumar
Software Reliability Improvement
• Reliability is improved when software faults which occur in the most frequently used
parts of the software are removed.
• Removing x% of software faults will not necessarily lead to an x% reliability
improvement.
• In a study, removing 60% of software defects actually led to a 3% reliability
improvement.
• Removing faults with serious consequences is the most important objective.
• Error prevention
• Fault detection and removal
• Measurements to maximize reliability.
✓ specifically the measures who support the first two activities.
Activities to Improve Software Reliability

17. Dr. Ajeet Kumar
Software Reliability Estimation
• Software reliability estimation determines current software reliability by
applying statistical inferences to the failure data obtained during system
testing or during system operation.
• Purpose of reliability estimation is to determine whether the applied reliability
model is good enough for the current software project.
• There are numerous software reliability estimation models available today.
• Software testing managers use these models to determine time and resources
requirements for software testing.
• Software managers use estimation models to determine a suitable MTTF
objective for delivery as well as resources required for maintaining software.

18. Dr. Ajeet Kumar
Software
Reliability
Estimation
Model

19. Dr. Ajeet Kumar
Some
Useful
Software
Reliability
Models
Jelinski &
Moranda (JM)
Model
Shooman Model
Musa’s Basic
Execution Time
Model
Goel-Okumoto
Model
Schneidewind’s
Model
Hyperexponential
Model
Schick-Wolverton
S-shaped
Reliability Growth
Model
Duane’s Model
Geometric Model
Musa- Okumoto
Logarithmic
Poisson
Littlewood-
Verrall Reliability
Growth Model

20. SOFTWARE SAFETY

21. Dr. Ajeet Kumar
• Safety: freedom from unacceptable risk.
• What is Safety? What are various kind of Safety applicable to Systems?
• Functional Safety, Occupational Safety, Fire-life Safety, Health Environment & Safety (HES), etc.
• Safety-Critical System: are the systems that is intended to achieve, on its own, the necessary level of safety
integrity for the implementation of the required safety functions.
• Safety Integrity: the ability of a safety-related system to achieve its required safety functions under all the
stated conditions and environment and within a stated period of time.
• Safety Integrity Level: a number which indicates the required degree of confidence that a system will meet
its specified safety functions with respect to systematic failures.
• Hazard: a condition that could lead to an accident. Each Hazard shall be characterized for some probability
of occurrence and its associated severity.
• Hazard Analysis: the process of identifying hazards and analysing their causes, and the derivation of
requirements to limit the likelihood and consequences of hazards to a tolerable level. Typical hazard
analysis techniques are PHA, SHA, IHA and O&SHA.
Safety Related Terms (H/W and S/W)
Software Safety is defined as: The probability of mishap-free software operation for a
specified period of time in a specified environment

22. Dr. Ajeet Kumar
• Both H/W & S/W should be developed as per the guidelines provided by the
Standards for the specific SILs ( Safety Integrity Level).
• The concept of SIL (1 to 4) is given by IEC 61508 which the then adopted by
other industries ( Railways, Automotive, Avionics etc..)
• Hardware systems are evaluated for both random and systematic failure.
• Software systems are evaluated for systematic failure.
• Software has no inherent failure rate, only fails systematically and therefore can
be evaluated systematically.
• Software can not be guaranteed to be 100% correct but the errors can be
minimized at various stages of development (Systematically).
• Requirement; Design; Coding; Test, etc.
Functional Safety: Safety Critical Software

23. Dr. Ajeet Kumar
• A functionally safe software is one that responds as expected for every set of inputs.
• During the S/W development faults may be introduced at any stage; testing ensure
their presence, faults not detected during testing remain in the system.
• There is no known way to prove the absence of faults in a complex safety-related
software.
• Many standards such as EN 50128, ISO 26262, DO 178 provides guidelines for
development of Safe and Reliable Software.
• EN 50128 provides guidelines for development of Safe Software for Railways
Systems.
• ISO 26262 provides guidelines for development of Safe Software for Automotive
Industries.
• DO 178 provides guidelines for development of Safe Software for Avionics
Industries.
23
Functional Safety for Safety Critical Software

24. Dr. Ajeet Kumar
Software in Railways: Safety Critical Application
• Earlier Railway Systems uses electronic
subsystems/components controlled mechanically
or manually.
• For better performance, flexibility and passenger
comfort; things are automated and software
controlled.
Few Years Back
Uses simple electronics subsystems
controlled mechanically or manually.
Now a days almost all the subsystem uses new electronic
control and monitoring system ( software based).
Now
There are a need for a safe and reliable software for other
industries as well. These lists ranges from a general
purpose software to safety critical software.
Automotive- Driverless Vehicles
Avionics
Smart Watches
Smart Phones
Medical Devices.

25. Dr. Ajeet Kumar
• Software bugs surface out under certain input conditions.
• It is not possible to test the system under all possible conditions.
• Example for Railway Signalling System – Interlocking at a 4-line station
• Tracks = 20 to 25; Crossovers = 4 to 6; Signal aspects = 30; Routes = 20
• Total number of conditions = 280
• An accelerated testing of the software for all possible conditions @ 1 per millisecond will take many years
• To ensuring complete software testing; 100% code is to test under all conditions.
• Functional testing does not ensure 100% coverage because many functions have error-
handling code, which can be difficult or impossible to stimulate using a fully integrated
application
Why Functional Testing Isn’t Enough for 100% Coverage

26. Dr. Ajeet Kumar
• System Reliability depends on Reliability of Hardware, Software and Human.
• Software failure causes most of system failure which might impact performance and safety
of the System.
• Majority of safety functions in many industrial applications are implemented and/or
controlled by software.
• Testing are used to ensure the Reliability and Safety of these systems.
• Exhaustive testing of these systems are not possible, therefore, systems must be designed
in such a way to prevent failure or ensure controlled behavior if failures arise .
• Software development must comply to applicable Safety Standards to meet the Safety
Requirements.
➢ So standard process such as EN 50128 for Railways shall be followed to minimize
systematic error in software.
➢ So standard process such as DO 26262 for Automotive shall be followed to minimize
systematic error in software.
➢ So standard process such as DO 178 D for Avionics shall be followed to minimize
systematic error in software.
Summary: Software Reliability & Safety

27. Any Question Pls…
Thank You!