What is software quality? How we can achieve it?

1. Quality assurance in modern software
development
Zahra Sadeghi
1

2. Resources
• Tripathy, Priyadarshi, and Kshirasagar Naik. Software testing and quality assurance:
theory and practice. John Wiley & Sons, 2011. (pdf available)
• Galin, Daniel. Software quality assurance: from theory to implementation. Pearson
education, 2004. (pdf available)
• A friendly introduction to software testing (available on github)
• https://github.com/wonsunahn/CS1632_Spring2020/blob/master/software-quality-
assurance-textbook.pdf
2

3. Administrative
• Homeworks (due Tuesday 11:59 p.m.)
• Research Projects (to be presented during the semester)
• Research and Development Project (one week after final exam, the exact time will be announced
later)
3

4. Today's agenda
• Software Quality Assurance vs Software Quality Control
• Software Quality Models
• Quality Assurance Methodology
• Quality control assistance tools
• Software Quality Testing
• Software Quality and User Experience
• Improving Software Quality with AI
4

5. Review: Basic
concepts
• Software: a set of instructions, data or programs used to
operate computers and execute specific tasks.
• System software to provide core functions such as operating systems,
disk management, utilities, hardware management and other operational
necessities.
• Programming software to give programmers tools such as text editors,
compilers, linkers, debuggers and other tools to create code.
• Programmers, or coders, write source code to program
computers for specific tasks like merging databases, processing
online orders, routing communications, conducting searches or
displaying text and graphics.
• Software engineers apply computer science
and mathematical principles to build software and systems
to solve problems. Have deep understanding of software
development, coding languages, and operating systems.
• Software developers working across functional teams to
transform consumer requirements into features, managing
development teams and processes, conducting software testing
and maintenance, reviewing customer expectations and end-
user experience
5

6. Software
Quality
(IEEE definition)
1. The degree to which a system, component, or
process meets specified requirements.
• Requirements: expected
business/product/technical characteristics
• internal factors such as readable and
modular code which can only be detected
by computer professionals who have access to
the source code.
2. The degree to which a system, component, or
process meets customer or user needs or expectations.
• Expectations: mainly end-user expectations
• external factors such as speed or ease of
use which can be detected by users
6

7. Quality Assurance
vs. Quality Control
• QC: to inspect an accomplished product to
ensure its alignment with the project scope
(product oriented)
• QA: includes management of the quality of
the whole process and aspects of software
development (process oriented)
7
Q: Which one is correct?​
A: QA is part of QC
B: QC is part of QA
being prepared for the
unexpected
Minimizing the chance of a
poor quality service

8. Quality Models
• McCall’s Quality Model.
• Boehm’s Quality Model.
• FURPS Quality Model.
• ISO 9126 Quality Model.
8

9. Important factors in
software quality
McCall’s Factor Model
9

10. Product
operation
factors
• Correctness –
• The extent to which a software meets its requirements specification (Accuracy of
output, Completeness of output information , Up-to-date of the information and Availability
of the information)
• E.g. The probability of missing data about a member, his attendance at club events, and his
payments will not exceed 1%
• Efficiency – (Size of developed software)/(Resources Used)
• refers to the number of resources such as time, personnel, equipment & effort required to
develop software.
• Q. Bid A consists of three communication lines of 28.8 KBPS between each branch and the
head office, whereas Bid B is based on two communication lines of the same capacity
between each branch and the head office. Which bid is more efficient?
• A: Bid B is more efficient than Bid A because fewer hardware resources are required.
• Integrity –
• The extent to which the software can control an unauthorized person from the accessing the
data or software (system security)
• E.g. to allow citizens access to GIS files through the Internet. The software requirements
include the possibility of viewing and copying but not inserting changes in the maps of their
assets as well as any other asset in the municipality’s area
• Reliability –
• The extent to which a software performs its intended functions without failure.
• E.g. The failure frequency of a heart-monitoring unit that will operate in a hospital’s intensive
care ward is required to be less than one in 20 years.
• Usability –
• The extent of effort required to learn, operate and understand the functions of the software
(deal with the quality of a user’s experience)
• E.g. Training a new employee to use the software will take no more than two days 10

11. Product
revision factors
• Maintainability –
• The effort required to detect and correct an error during maintenance
phase.
• the efforts to identify the reasons for software failures, to correct the
failures, and to verify the success of the corrections.
• This factor’s requirements refer to the modular structure of software, the
internal program documentation, and the programmer’s manual
(e.g.1) The size of a software module will not exceed 30 statements.
(e.g.2) The programming will adhere to the company coding standards and
guidelines.
• Flexibility –
• The effort needed to improve and adapt operational software program to
different customers/various activities.
(e.g.1) The software should be suitable for teachers of all subjects and all
school levels (elementary, junior and high schools).
• Testability –
• The effort required to verify a software to ensure that it meets the
specified requirements.
(e.g.1) One testability Software quality factors requirement demanded is to
develop a set of standard test data with known system expected correct
reactions in each stage.
This standard test data is to be run every morning, before production
begins, to check whether the computerized unit reacts properly.
11

12. Product
transition
factors
• Portability –
• The effort required to transfer a program from one platform to
another.
• (E.g.) A software package designed and programmed to
operate in a Windows environment is required to allow low-
cost transfer to Linux
• Re-usability –
• The extent to which the program’s code can be reused in other
software applications.
• (E.g.) some of the swimming pool software modules for the
pool should be designed and programmed in a way that will
allow its reuse in the spa’s future software system
• Interoperability – inter + operable
• The effort required to integrate two systems with one another
(creating interfaces with other software systems).
• (E.g.) In a hospital administration system, the records of
patients are exchanged between 1 department to other
department. So department can be linked to an application.
Details of the patient get exchanged between 1 application to
another application without any prior notice.
12

13. Important factors in
software quality
Bohem's model
13

14. Assignment 1
• Define all the factors in Bohem's model by
providing an example for each.
14

15. Quality
Assurance
Methodology
(PDCA)
• problem solving and business-solving
• PLAN : Identify the problem. Establish objective
• DO : Develop process and do testing. Carryout the
plan. Document problems and unexpected
observations.
• CHECK : the data and results are evaluated. Reviews
tests, analyzes result.
• ACT : Enhance the procedure. Actions are taken from
the knowledge gained.
• The changes are adopted and the best solution is
implemented by running through the cycle again.
Walter A. Shewhart: Father of statistical quality control
1891-1967 15

16. Quality
Management
Tools
• Ishikawa Diagram
• Pareto Chart
• Control Chart
16

17. Ishikawa Diagrams: cause-
and-effect, fishbones
o identifies many possible
causes for an effect
or problem.
o Finds which
resources/actions are
required at specific times.
o determine which issues
have to be addressed in
order to gain or avoid a
particular event.
• Manpower - Anything related to people that are involved with the process
relevant to the problem
• The training, skill, and attitude of the employees or workers​
• Machines - all facilities and equipment that may be causing the problem
• Maintenance of machines, whether upgrades to better technology
is needed​
• Materials - All supplies and resources used in the production or process.
• Are raw materials and inputs properly labeled, stored, and of high
quality.
• Measurement -Techniques and tools used to measure quality standards
• Are methods of measurement and control correct and accurate? Do
they need to be adjusted?​
• Mother Nature - the external factors of the production area or the
environmental conditions such as temperature, humidity, pressure
• Method - Does the production process have the most efficient number of
steps, are there bottlenecks, is it overly complex and error-prone?​
17
Kaura Ishikawa
1915-1989

18. Ishikawa
Diagrams
• E.g. Identify the reasons for preparing a bad-quality
burger.
Q: What are the disadvantages?
A:
- The brainstorming process can produce
irrelevant potential causes along with
relevant ones, which can result in
confusion and a time drain.
- Complex issues may lead to a messy
diagram
- Based on opinion rather than evidence;
it needs testing to prove results
18

19. Assignment 3
• What are the 4 S's and 8 P's Ishikawa diagrams?
• Create an Ishikawa diagram for software defect
detection
19

20. Pareto Chart
• Pareto principle, which is the origin of the
80/20 rule.
• This premise states that approximately
80% of outcomes originate from 20% of
causes for many study areas.
• you probably use 20% of the applications
on your PC about 80% of the time. And,
you likely wear 20% of your clothes 80% of
the time.
• shows the ordered frequency counts or
cost (time/money) for levels of a nominal
variable in a bar chart.
• helps people decide which problems to
solve first.
• the chart visually depicts which situations
are more significant.
20
Vilfredo Pareto
1848-1923
• Pareto charts can also include a line for the cumulative frequency
e.g. the relative frequency of
reasons for arriving late at work

21. Control chart
• The control chart is a graph used to study
how a process changes over time.
• Data are plotted in time order.
• A control chart always has a central line for
the average, an upper line for the upper
control limit, and a lower line for the lower
control limit.
• Draw conclusion about in-control and out-
of-control events
21

22. The Cost of Quality in
Software Development
(COQ)
• The COQ is the cost incurred to check
that the Product conforms to the
client’s requirement efficiently and
satisfactorily and meets all the agreed
and desirable quality points.
• COQ is extra expenses, beyond
production costs, to ensure the quality
end-product
Q: Which types of costs deals with quality assurance?
A: Prevention costs
22
• External failure costs – linked to defects the customer finds post-sale,
e.g. costs to process customer complaints, returns, warranty claims.
• Internal failure costs – linked to defects found before selling the
product to customers, e.g. re-work, re-testing, bug fixing, re-design.
• Appraisal costs – incurred to determine conformance to quality
requirements, e.g. measurements, audits, evaluations, inspections,
testing.
• Prevention costs – incurred to prevent bad quality, e.g. quality
planning, project management, feature review, product review, team
training.

23. Software quality
testing
23
 Software quality testing
evaluates a software while
helping a manufacturer produce
high quality software
 Functional testing: focuses on checking the application’s
functionality against a set of requirements or
specifications.
o Non-functional testing checks the behavior of an Application.
o Non- functional testing is done based on the customer
expectation and Performance requirement.
• Black box testing:
• check if the functionality works as planned
• White box testing:
• thoroughly review the code of the software
• Grey box testing
• a mixture of the two kinds of where the testers of the
software have knowledge of the internal data structures and
design tests based on those algorithms but execute those
tests at the user or the black box level

24. 24
Functional testing
Non-Functional testing
• Performance testing
• software system responds to requests promptly (response time, failure points)
• Usability testing
• measures the quality of a user’s experience
• Security testing
• applications are secure and handle data correctly
• to identify any potential security risks or threats and to ensure that the product is not vulnerable to
hacking, data breaches, or other types of security issues.

25. Unit testing
Q: Is it a black box testing or white
box testing?
• is a level of software testing where
individual units/components of a
software are tested.
• Unit testing is testing the smallest
testable unit of an application.
• It is done during the coding phase by
the developers.
assert statement allows you to write sanity
checks in your code.
assert sum([1, 2, 3]) == 6, #"Should be 6"
A: White box
25

26. Integration
testing (I&T)
• verify whether the combined units work well together as a
group.
• is aimed at detecting the flaws in the interactions
between the units within a module.
• checks that components in your application operate with
each other.
• e.g: Interfaces of the software modules with the database
could be erroneous
• The bottom-up integration testing starts with unit tests,
successively increasing the complexity of the
software modules under test.
• The top-down method takes the opposite approach, focusing
on high-level combinations first and examining
the simple ones later.
Q: Is it a black box testing or white
box testing?
A: gray box 26

27. Assignment 4
• Compare top-down with bottom-up approaches
with a practical example
• What are the pros and cons of each approach and
which one would you apply for your software
development test?
27

28. System
testing
• System testing entails testing the whole
system.
• All of the modules/components are linked
together to see whether the system
performs as planned.
• With the use of a specification document,
this testing assesses the system's
functionality from the perspective of the
user.
Q: Is it a black box testing or white
box testing?
A: black box
28

29. Acceptance
testing
• verifying that a solution works for the user
(UI and performance)
• This final step helps the team decide if the
product is ready to be shipped or not.
• The acceptance stage might be followed by
an alpha and beta testing, allowing a small
number of actual users to try out the
software before it is officially released.
• Alpha testing: performed by the testers
who are part of the organization, in other
words: internal employees.
• Beta testing: Users can freely use the
application and then they are encouraged
to give feedback about their experience.
Q: Is it a black box testing or white
box testing?
A: black box
29

30. Manual vs Automated testing
Manual testing Automated testing
Time consuming and expensive Save time and money, faster testing
Low productivity High productivity
Not very accurate, error prone Highly accurate
30
• Manual: Testers manually scour through different features of
a website or app in order to identify bugs, errors, anomalies,
and the like.
• test cases are executed manually (by a human)
• Automated: Testers set up frameworks and create test
scripts that automate user actions required for testing a
website or app.
• test cases are executed with the assistance of tools,
scripts, and software

31. Customer satisfaction and software quality
31

32. Software Quality and User eXperience
• User experience impacts quality assurance
• Quality of a user's experience is known as usability
• refers to interacting with products such as ease of
learning, efficiency of use, and user satisfaction.
•User Interface design is all about how an application looks and feels.
•focuses on making sure that the user interface is both effective and easy to use.
•color scheme, layout (appearance)​
•User eXperience design focuses on making sure that the user experience is as smooth and intuitive as possible.​
•users’ needs and goals, navigation, and overall functionality​
32

33. Human Computer
Interface (HCI)
• The primary objective of HCI is to
design systems that make them
accessible, usable, efficient, and safe
for anyone and everyone.
• Cognitive HCI: humanize the human–
computer interface, including speech
recognition, speech to text, question
& answer, imaging understanding, etc.
wearable systems (smart watches/glasses/clothing), voice
assistants, health trackers, and smart TVs
33

34. Brain Computer Interface
(BCI)
• BCI is a system that
determines functional intent - the
desire to change, move, control, or
interact with something in your
environment - directly from your brain
activity.
• BCIs allow you to control an
application or a device using only your
mind.
1- A device to measure brain activity.
2- A computer to process and
analyze the recorded brain activity.
3- An application/device to control. 34
Accessibility design is a branch of UX design that
focuses on making sure that individuals with
disabilities can also use a software.

35. Research project
(optional)
• Identify a tool/gadget/device
with HCI/BCI that is currently in
use and introduce it.
• Explain the components of the
architecture.
• What are the shortcomings of
this application and how you
would improve them?
• Implement an HCI/BCI
application
35

36. Increasing
software
quality with
AI
• write programs or even translate from one
programming language to another.
• Accelerate application development with automatic
code recommendations based on the code and
comments in your IDE.
• perform tasks like offering assistance in auto-
generating documentation and finding code snippets
faster.
• Empower developers to responsibly use artificial
intelligence (AI) to create syntactically correct and
secure applications.
• OpenAI Codex is capable of interpreting simple commands in
natural language and carrying them out for the programmer.
• Amazon codeWhisperer
• ChatGPT By taking advantage of transformers, ChatGPT can sound
very human-like. It has a wider scope of responses and can
even generate code.
Coding with AI
36

37. Research project
(optional)
• Investigate and introduce one of
the mentioned automated
coding with AI tool
• Installation procedure
• Programming
• Develop an application with this
AI tool
37

38. Increasing
software
quality with
AI
• Artificial Intelligence will be helpful to
automate the processes in an efficient way
instead of a human tester.
• completing repetitive tasks with better
accuracy and less time span.
• Maintaining a large number of test scripts gets
difficult as an application grows.
Testing with AI
38
• Testim uses artificial intelligence and machine learning to
speed-up the authoring, execution, and maintenance of
automated tests.
• Applitools is an AI-powered visual UI testing and
monitoring software.
• Functionize is a cloud-based automated testing technology
for test creation, execution, maintenance, and analytics,
with capabilities for functional, visual, performance testing

39. Research project
(optional)
• Introduce an AI tools for testing
• Explain its features and
functionalities
• Use this tool for debug and
testing your software
39

40. Increasing
software
quality with
AI
• As AI models becomes more advanced, humans are challenged to
comprehend and retrace how the algorithm came to a result.
• It's not known what exactly is happening inside them or how the
AI algorithm arrived at a specific result.
• Explainable AI is crucial for an organization in building trust
and confidence when putting AI models into production.
• Help user understand how to change the outcome
• Explainability can help developers ensure that the system
is working as expected, meeting regulatory standards
Adding
understandability
with XAI
• AIX360 from IBM Research
• help debug models, where systems can explain what
they’re doing.
• This includes training highly optimized, directly
interpretable models, as well as explanations of black-
box models and visualizations
• Rulex Explainable AI
• creates predictive models in the form of first-order conditional
logic rules that can be immediately understood and used by
everybody. 40

41. Research project
(optional)
• Investigate and introduce an XAI
toolkit
• Show its performance by an
example
• Develop an XAI tool
41

42. Next lecture
• Project management
• Project planning and scheduling
• Risk management
• Resources:
• Edition, PMBOK–Sixth. "A guide to the project management body of
knowledge." Project Management Institute. Pensylvania (2018).
• Tom, Kendrick. "Identifying and Managing Project Risk." American
Management Association 1601 (2003).
42