Do you identify and separate different attributes for different goals? For example, do you use have specific metrics for quality but different ones for runtime, usability, system quality attributes? We welcome you to review Shilpa's insights here. Please visit our Xbosoft blog as well to gain additional insights on how to drive business decisions with software quality.

1. www.xbosoft.com
Webinar
March 19 2014 11.00 AM EST
You are at the right
place, it’s almost
time….10 minute
countdown

2. 2

3. WELCOME
XBOSOFT Software Quality
Knowledge Center is proud to
partner with and introduce
Shilpa Dadhwal
CEO and Founder
SQE Labs, Inc.

4. Welcome to the session on
‘Measuring Software Quality –
Its Economic Value’
Shilpa Dadhwal
Founder & Director
SQE Labs Inc.
Skype ID : shilpa2d
Email ID : shilpa@sqelabs.com
Presented By:

5. On the call
Shilpa Dadhwal, Philip Lew and Jan Princen
Moderator: Carolyn Choh
Shilpa Dadhwal,
Founder and CEO,
SQE Labs Inc
Philip Lew
CEO,
XBOSoft
Jan Princen
Managing Director,
XBOSoft

6. Agenda
• Introduction XBOsoft
• Software Quality – Quick review of Day 1
• Moving on to Day 2
– Looking at Quality – what attributes to measure?
– Implications for Senior Management
– Implications for Project Management
• Benefits
• QA & Next Sessions at 11 am -12 noon EST
• Today’s Meeting March 19th
https://www3.gotomeeting.com/register/19759
1286
• Tomorrow’s meeting March 20th
https://www3.gotomeeting.com/register/886356774

7. • Specialize in: Software Quality
Improvement
• Quality Process Improvement &
Testing
• American Company
• Located in: San Francisco, Amsterdam
& Beijing
• Over 100 Software Quality Experts
XBOSoft Introduction

8. “Proven Results”
“Our products are very complex. We expected a one
year ramp up time, but XBO was able to contribute to
our release cycles in just 6 months. ."
- Development Vice President, Hyperion-
Oracle
“
"The results are very encouraging. Thanks a lot for all the effort put
into this project. We really appreciate your work."
- Manager, Quality Assurance, Autodesk
"Everything you've done is impressive and we are quite pleased with the
quality of the test cases, well done!
So keep up the good work."
- QA Director, CA

9. Welcome to the Session 2 on
‘Measuring Software Quality –
Its Economic Value’
Shilpa Dadhwal
Founder & Director
SQE Labs Inc.
Skype ID : shilpa2d
Email ID : shilpa@sqelabs.com
Presented By:

10. SQE Labs offers an independent Software Testing and Quality
Assurance services and is committed to improving quality by
providing superior QA solutions to customers
About USA Specialist Independent Software Testing Solutions Provider
Team of Software Quality Engineering professionals having strong
Techno-functional knowledge
Provides services in the areas of Outsourced Software Testing,
Software Testing Consultancy and Software Testing Trainings
Having Offices in Texas (US) and Chandigarh (India)

11. What is Software Quality?
What is its Economic Value?
How to mange Software Quality?
What are its Attributes?
Why do we need to measure it?
How do we measure it?
Issues while measuring it?
Guidelines for measuring it?
A Specialist Independent Software Testing Solution Provider Questions in
Mind
Session 1
Session 2
Session 3

12. 1. Define what quality means to you, for your software? (Give the highest priority
parameter)
‘High Reliability when Operating’
2. How can you measure this factor in your software?
No. of crashes encountered while working on the system for specified no. of hours
3. Does this require any corrective or preventive action plan?
Yes
4. What action could be taken to improve it?
Testing of Memory Leaks in the application
5. How can you keep track of the same in any future product?
Record the no. of crashes encountered while running the application for
specified no. of hours
Case Study from
Session 1
A Specialist Independent Software Testing Solution Provider

13. 1. What is a METRIC?
2. What are DIFFERENT TYPES of Metric?
3. What are the desirable CHARACTERISTICS of a Metric?
4. What are the ATTRIBUTES of Software Quality?
5. How to NORMALIZE Metrics?
A Specialist Independent Software Testing Solution Provider
Agenda

14. “When you can measure what you are speaking
about and express it in numbers you know
something about it;
but when you cannot measure it, when you cannot
express it in numbers, your knowledge is of a
meagre and unsatisfactory kind”.
William Thomson (Lord Kelvin)
QuoteA Specialist Independent Software Testing Solutions Provider

15. PollA Specialist Independent Software Testing Solutions Provider
 How much time will you take to integrate this new module with
the existing module?
• Much less
 In Project A , 100 defects have been logged but in Project B, 200
defects have been logged. So which Project is better?
• Project B
 How is the Software Quality of your product?
• Good
Can you as a Manager , make any useful decisions from
such subjective information?
1) Yes
2) No

16. Measure: Quantitative indication of the extent, amount, dimension,
or size of some attribute of a product or process. A single data point
Metrics: The degree to which a system, component, or process
possesses a given attribute. Relates several measures (e.g. average
number of errors found per person hour)
Direct Metrics: Immediately measurable attributes (e.g. line of code,
execution speed, defects reported)
Indirect Metrics: Aspects that are not immediately quantifiable (e.g.
reliability, productivity)
Common
Terminology
A Specialist Independent Software Testing Solutions Provider

17. Products
• Quantify the characteristics of the product being developed
e.g. Size, reliability
Process
• Quantify the characteristics of the process being used to
develop the software
e.g. Efficiency of fault detection, Effectiveness of Defect Removal
Project
• Quantify the characteristics of the project and its execution
e.g. Staffing pattern over the lifecycle of the project, Productivity
Software Metric
Classification
A Specialist Independent Software Testing Solutions Provider

18. Product metrics are measures of the software product at any stage
of its development, from requirements to release.
Focus on the quality of deliverables
Product metrics are combined across several projects to produce
process metrics
Metrics for the product:
o The complexity of the Software design
o Size of the final program
o Number of pages of documentation produced
o Defect density
o Customer Satisfaction index
Product MetricsAn Independent Software Testing Solution Provider

19. Focus on quality achieved as a consequence of a repeatable or
managed process.
Process Metrics are measures of the software development process,
such as
o Overall development time
o Average delay of milestone completion
Process metrics are collected across all projects and over long periods
of time
Their intent is to provide indicators that lead to long-term software
process improvement
It is defined mostly at a Strategic level
Process MetricsAn Independent Software Testing Solution Provider

20. To improve any process,
Measure specific attributes of the process
Derive meaning metrics from these attributes
Use these metrics to provide indicators
The indicators lead to a strategy for improvement
Process
Improvement
A Specialist Independent Software Testing Solutions Provider

21. Project Metrics are measures of Software project and are used to
monitor and control the project
Used by a project manager and software team to adapt project work
flow and technical activities:
• Minimize the development schedule by making the
necessary adjustments to avoid delays and mitigate
problems
• Assess product quality on an ongoing basis and modify the
technical approach to improve quality
Used in estimation techniques and other technical work
Project MetricsA Specialist Independent Software Testing Solutions Provider

22. Metrics collected in the past are used as a basis from which effort
and time estimates are made for the current project
Metrics:
o Effort or time per SE task
o Errors uncovered per review hour
o Scheduled vs. actual milestone dates
o Number of changes and their characteristics
o Distribution of effort on SE tasks
Tactical and Short Term.
Project MetricsAn Independent Software Testing Solution Provider

23. METRICS should be
– RELIABLE (free from random error)
– REPEATABLE (same entity, same environment, same visitors and same
evaluator)
– REPRODUCIBLE (same entity, same environment, same visitors but
different evaluator)
– AVAILABLE (constraint conditions)
– INDICATIVE OF IMPROVEMENT (of the entity quality)
– CORRECT (objective, impartial and precise)
– MEANINGFUL (about the entity behavior or quality characteristics).
Desirable characteristics
of Metrics
An Independent Software Testing Solution Provider

24. A Metric is a qualitative or quantitative measurement of a specified
Software Quality Attribute, for measuring Software Quality
Software Quality Metrics are subset of Software Metrics, that focus
on quality aspects of product, process and project
Thus, in order to define Metrics for measuring Software Quality, we
must know which Quality Attributes needs to be measured
The Quality Requirement Specification gives quantitative and
qualitative values for these attributes that must be satisfied by the
executable system
Measuring
Software Quality
An Independent Software Testing Solution Provider

25. Software Quality attributes is a property of a software that can have
some qualitative or quantitative value and can be measured or observed.
Categories of Software Quality Attributes are :
1. Design Quality Attributes
2. Runtime Quality Attributes
3. System Quality Attributes
4. User Quality Attributes
5. Non-runtime Quality Attributes
6. Architecture Quality Attributes
7. Business Quality Attributes
Software Quality
Attributes
A Specialist Independent Software Testing Solutions Provider

26. Conceptual Integrity
• Defines the consistency and coherence of the overall design
• Includes the way the components or modules are designed
Metric – i) Mean Time to Change
Maintainability
• Ability of the system to undergo changes with a degree of ease
Metric – i) Mean Time to Change
Reusability
• Defines the capability for components and subsystems to be suitable for
use in other applications
Metric – i) Implementation time
Design Quality
Attributes
An Independent Software Testing Solution Provider

27. Interoperability
• Ability of a system to operate successfully by communicating and exchanging
information with other external systems written and run by external parties
Metric – i) Modularity
Manageability
• Defines how easy is it for system administrators to manage the applications
Metric – i) Implementation time
Reliability
• Ability of the system to remain operational over time
Metric – i) Mean Time to Failure
Scalability
• Ability of the system to either handle increases in load without impact on the
performance of the system or the ability to be readily enlarged
Metric – i) Horizontal Scalability
Runtime
Quality Attributes
An Independent Software Testing Solution Provider

28. Performance
• Indication of the responsiveness of a system to execute any action
Metric – i) Throughput
Security
• Capability of a system to prevent malicious or accidental actions outside of the
designed usage
Metric – i) No. of Security defects logged per KLOC
Availability
• Portion of the time the system is functional and working
Metric- i) Total Unavailability = Measured as a percentage of the total system
downtime over total no. of hours of service in a year period
Runtime
Quality Attributes
A Specialist Independent Software Testing Solutions Provider

29. Supportability
• Ability of a system to provide information helpful for identifying and
resolving issues when it fails to work correctly
Metric – i) Time taken to resolve an issue encountered in the software
Testability
• Measure of how easy it is to create a test criteria for a system and its
component
Metric – i) Modularity
System
Quality Attributes
An Independent Software Testing Solution Provider

30. Usability
• Defines how well the application meets the requirements of the user
and consumer by being intuitive
Metric - i) Learnability
User
Quality Attributes
An Independent Software Testing Solution Provider

31. Portability
• Ability of a system to run under different computing environments
Metric – i) Install ability
Integrity
• Ability to make the separately developed components of the system work
correctly together
Metric – i) Access Control
Reusability
• Degree to which existing applications can be reused in a new application
Metric- i) Implementation time
Modifiability
• Ease with which a software system can accommodate changes to its software
Non-Runtime
Quality Attributes
A Specialist Independent Software Testing Solutions Provider

32. Correctness
• Accountability for satisfying all requirements of the system
Metric - Defects per FP/LOC
Conceptual Integrity
• Integrity of the overall structure that is composed from a number of
small architectural structures
Architecture
Quality Attributes
An Independent Software Testing Solution Provider

33. Cost
• Cost of a system with respect to time to market, expected project
lifetime
Metric – i) Time to Market
Marketability
• Use of the system with respect to market competition
Metric – i) Price Earning ratio
Business
Quality Attributes
An Independent Software Testing Solution Provider

34. In Project A , 100 defects have been logged. But in Project B, 200 defects have been
logged.
So which Project is better?
Can we compare the two projects, just by looking at the above data?
Additional information required:
KLOC for Project A = 1000
KLOC for Project B = 10,000
Defect Density of Project A = 100/1000= 0.1defects/KLOC
Defect Density of Project B = 200/10000= 0.02 defects/KLOC
Thus, in Project A more defects have been logged per KLOC as compared to Project B
Normalization of
Metrics
An Independent Software Testing Solution Provider

35. How does an organization combine metrics that come from different
individuals or projects?
Depend on the size and complexity of the project
Normalization: compensate for complexity aspects particular to a
product
Normalization approaches:
Size oriented (lines of code approach)
Function oriented (function point approach)
Normalization of
Metrics
An Independent Software Testing Solution Provider

36. Case Study of
Metrics Program
An Independent Software Testing Solution Provider
• Motorola followed Goal/Question/Metric paradigm
• Goals were identified, questions were formulated in quantifiable terms and
metrics were established
• Few Examples are as follows:
• Goals:
• Goal 1: Increase Software Reliability
• Goal 2: Reduce Cost of Non-Conformance
• GOAL 1: INCREASE SOFTWARE RELIABILITY
• Question: What is the rate of software failures and how does it
change over time?
• Metric: Failure Rate = Number of failures/Execution time
• GOAL 2: REDUCE COST OF NON-CONFORMANCE
• Question: What is the cost to fix post –release problems during
the month?
• Metric: Cost of fixing Problems = Dollar cost associated with
fixing post-release problems within a month

37. To be continued in Session 3
Next…A Specialist Independent Software Testing Solution Provider

38. What is Software Quality?
What is its Economic Value?
How to mange Software Quality?
What are its Attributes?
Why do we need to measure it?
How do we measure it?
Issues while measuring it?
Guidelines for measuring it?
A Specialist Independent Software Testing Solution Provider Questions
Answered
Session 1
Session 2
Session 3

39. Quality attributes are measurable or observable characteristics of
products and processes.
The quality requirements specification gives quantitative or
qualitative values for these attributes that must be satisfied by the
executable system.
Software Quality can be measured only if you know which
Software Quality Attribute should be measured and which metric
should be used for measuring the same.
A Specialist Independent Software Testing Solution Provider
Few Remarks

40. Thank YouA Specialist Independent Software Testing Solutions Provider
India Office
SQE Labs Inc.
116/8, Mohali (Chandigarh)
India – 160062
www.sqelabs.com
US Office
SQE Labs LLC,
6533, Preston Rd, Plano,
TX - 75024
Contact me : shilpa@sqelabs.com

41. Follow: @XBOSoft
Questions: services@xbosoft.com
408-350-0508

42. Thank You!
See you tomorrow
Tomorrow’s meeting March 20th
https://www3.gotomeeting.com/register/886356774
Shilpa Dadhwal,
Founder and CEO,
SQE Labs Inc
Philip Lew
CEO,
XBOSoft
Jan Princen
Managing Director,
XBOSoft