This document discusses models for predicting customer perceptions of software quality based on factors collected within the first three months of installation. Logistic regression is used to model rare, high-impact software failures based on variables like system size, software upgrades, operating system, etc. Linear regression is used to model frequent, low-impact customer interactions like calls based on similar predictor variables. The models found most predictors to be statistically significant due to the large sample size.

1. Predictors of Customer Perceived
SOFTWARE
QUALITY
presented by
Nicolas Bettenburg

2. Software Quality
matters!
01 / 28

3. Imagine the Product
Does NOT Satisfy
a Customers Needs ...
02 / 28

4. The
company suffers
•Maintenance costs
•Additional expenses
•Missed business opportunities
03 / 28

5. Predict customerʼs
experiences within the
first 3 months!
04 / 28

6. What are the
factors?
05 / 28

7. software platform hardware install location
software updates system size missing information
deployment issues usage patterns service contract
06 / 28

8. Operating System
System Size
Predictors
Software
Ports Upgrades
Deployment
Time

9. Use Predictors
to form Models
13 / 28

10. Software Failure
Rare, high-impact problems
resulting in a software change
use logistic regression.
Customer Interactions
Frequent, low-impact problems,
resulting in a customer call
use linear regression.
14 / 28

11. 15 / 28

12. Logistic Regression
xi β
e
P (Yi = 1|xi ) =
1+exi β
16 / 28

13. Logistic Regression
xi β
e
P (Yi = 1|xi ) =
Binary
1+exi β
Response
Variable
16 / 28

14. Logistic Regression
xi β
e
P (Yi = 1|xi ) =
Binary
1+exi β
Response Predictor
Variable Variable
16 / 28

15. Logistic Regression
xi β
e
P (Yi = 1|xi ) =
Binary
1+exi β
Response Predictor Logistic Model
Variable Variable for one predictor
Variable
16 / 28

16. Logistic Regression
Failure Report
System Size
17 / 28

17. Logistic Regression
Failure Report
System Size
18 / 28

18. Logistic Regression
19 / 28

19. Logistic Regression
Beta Coefficient
19 / 28

20. Logistic Regression
Beta Coefficient Significancy Measures
19 / 28

21. Software Failure Model
5.1.1 Modeling software failures
sys
Estimate Std. Err. z-value Pr(>|z|) sys
(Intercept) −5.26 0.64 −8.18 3 ∗ 10−16 tan
log(rtime) −0.30 0.03 −8.85 < 2 ∗ 10−16 as
Upgr 1.38 0.15 9.01 < 2 ∗ 10−16
OX −1.18 0.17 −6.75 2 ∗ 10−11 ah
WIN 1.01 0.34 2.98 0.003 ife
log(nP ort) 0.36 0.08 4.37 10−5 bo
nP ortN A 2.03 0.58 3.49 5 ∗ 10−4
LARGE 0.52 0.20 2.67 0.01 cau
Svc 0.57 0.18 3.11 .002 fer
US 0.52 0.27 1.92 0.05 the
a s
Table 1: Software failure regression results. tiv
20 / 28
ag

22. Software Failure Model
5.1.1 Modeling software failures
sys
Estimate Std. Err. z-value Pr(>|z|) sys
(Intercept) −5.26 0.64 −8.18 3 ∗ 10−16 tan
log(rtime) −0.30 0.03 −8.85 < 2 ∗ 10−16 as
Upgr 1.38 0.15 9.01 < 2 ∗ 10−16
OX −1.18 0.17 −6.75 2 ∗ 10−11 ah
WIN 1.01 0.34 2.98 0.003 ife
log(nP ort) 0.36 0.08 4.37 10−5 bo
nP ortN A 2.03 0.58 3.49 5 ∗ 10−4
LARGE 0.52 0.20 2.67 0.01 cau
Svc 0.57 0.18 3.11 .002 fer
US 0.52 0.27 1.92 0.05 the
a s
Table 1: Software failure regression results. tiv
20 / 28
ag

23. Software Failure Model
5.1.1 Modeling software failures
sys
Estimate Std. Err. z-value Pr(>|z|) sys
(Intercept) −5.26 0.64 −8.18 3 ∗ 10−16 tan
log(rtime) −0.30 0.03 −8.85 < 2 ∗ 10−16 as
Upgr 1.38 0.15 9.01 < 2 ∗ 10−16
OX −1.18 0.17 −6.75 2 ∗ 10−11 ah
WIN 1.01 0.34 2.98 0.003 ife
log(nP ort) 0.36 0.08 4.37 10−5 bo
nP ortN A 2.03 0.58 3.49 5 ∗ 10−4
LARGE 0.52 0.20 2.67 0.01 cau
Svc 0.57 0.18 3.11 .002 fer
US 0.52 0.27 1.92 0.05 the
a s
Table 1: Software failure regression results. tiv
20 / 28
ag

24. Software Failure Model
5.1.1 Modeling software failures
sys
Estimate Std. Err. z-value e!
Pr(>|z|)
as
sys
(Intercept) −5.26 0.64 −8.18 3 ∗e −16
el 10 tan
rr
log(rtime) −0.30 0.03 −8.85ajo 2 ∗ 10−16
< as
Upgr 1.38 a m < 2 ∗ 10−16
0.15 to 9.01
OX −1.18 0.17e −6.75
ra d 2 ∗ 10−11 ah
WIN 1.01 u pg 0.34 2.98 0.003 ife
log(nP ort) s t to
0.36 0.08 4.37 10−5 bo
fir
nP ortN Athe 2.03 0.58 3.49 5 ∗ 10−4
ʼt
LARGEbe 0.52 0.20 2.67 0.01 cau
d on Svc 0.57 0.18 3.11 .002 fer
US 0.52 0.27 1.92 0.05 the
a s
Table 1: Software failure regression results. tiv
20 / 28
ag

25. Linear Regression
E(log(Yi )) = xi β
21 / 28

26. Linear Regression
E(log(Yi )) = xi β
Number of
Customer Calls
21 / 28

27. Linear Regression
E(log(Yi )) = xi β
Number of Predictor
Customer Calls Variable
21 / 28

28. Linear Regression
# Customer Calls
System Size
22 / 28

29. Linear Regression
# Customer Calls
System Size
23 / 28

30. nician dispatches, and alarms within the first three months of in-
stallation using linear regression. For example, in the case of calls,
Customer Interactions
the response variable Y calls is the number of calls within the first
2000
three months of installation transformed using the log function to
make errors more normally distributed. The predictor variables, xi ˜
1500
Model
are described in detail in section 4. The model is:
Calls
1000
E(log(Yicalls )) = xT β
˜i
500
5.2.1 Modeling customer calls
0
2003.6
Estimate Std. Err. t value Pr(>|t|)
(Intercept) 0.35 0.04 7.90 3 ∗ 10−15
log(rtime) −0.08 0.00 −27.72 < 2 ∗ 10−16 Figu
Upgr 0.73 0.02 46.78 < 2 ∗ 10−16
OX 0.13 0.01 9.62 < 2 ∗ 10−16 The two tren
WIN 0.75 0.03 25.73 < 2 ∗ 10−16 flow of calls ca
log(nP ort) 0.10 0.01 16.82 < 2 ∗ 10−16 itations we do
nPortNA 0.39 0.04 10.80 < 2 ∗ 10−16 calls for new a
LARGE 0.30 0.01 20.78 < 2 ∗ 10−16
Svc 0.28 0.01 23.06 < 2 ∗ 10−16 6. VALID
US 0.41 0.01 28.99 < 2 ∗ 10−16 It is importa
that results refl
Table 3: Number of calls regression. R2 = .36. of the data coll
We inspecte
process and int
24 / 28
Most predictors are statistically significance due to large sample curacy. Throu

31. nician dispatches, and alarms within the first three months of in-
stallation using linear regression. For example, in the case of calls,
Customer Interactions
the response variable Y calls is the number of calls within the first
2000
three months of installation transformed using the log function to
make errors more normally distributed. The predictor variables, xi ˜
1500
Model
are described in detail in section 4. The model is:
Calls
1000
E(log(Yicalls )) = xT β
˜i
500
5.2.1 Modeling customer calls
0
2003.6
Estimate Std. Err. t value Pr(>|t|)
(Intercept) 0.35 0.04 7.90 3 ∗ 10−15
log(rtime) −0.08 0.00 −27.72 < 2 ∗ 10−16 Figu
Upgr 0.73 0.02 46.78 < 2 ∗ 10−16
OX 0.13 0.01 9.62 < 2 ∗ 10−16 The two tren
WIN 0.75 0.03 25.73 < 2 ∗ 10−16 flow of calls ca
log(nP ort) 0.10 0.01 16.82 < 2 ∗ 10−16 itations we do
nPortNA 0.39 0.04 10.80 < 2 ∗ 10−16 calls for new a
LARGE 0.30 0.01 20.78 < 2 ∗ 10−16
Svc 0.28 0.01 23.06 < 2 ∗ 10−16 6. VALID
US 0.41 0.01 28.99 < 2 ∗ 10−16 It is importa
that results refl
Table 3: Number of calls regression. R2 = .36. of the data coll
We inspecte
process and int
24 / 28
Most predictors are statistically significance due to large sample curacy. Throu

32. nician dispatches, and alarms within the first three months of in-
stallation using linear regression. For example, in the case of calls,
Customer Interactions
the response variable Y calls is the number of calls within the first
2000
three months of installation transformed using the log function to
make errors more normally distributed. The predictor variables, xi ˜
1500
Modelare described in detail in section 4. The model is:
Calls
1000
E(log(Yicalls )) = xT β
˜i
500
5.2.1 Modeling customer calls
ly!
0
Estimate Std. Err. t value ra te 2003.6
uPr(>|t|)
(Intercept) 0.35 0.04 7.90 acc3 ∗ 10−15
log(rtime) −0.08 ted
0.00 −27.72 < 2 ∗ 10−16
ic 46.78 < 2 ∗ 10−16 Figu
Upgr
OX
0.73
red 9.62 < 2 ∗ 10−16
0.02
0.13 e p0.01 The two tren
WIN
ca n b 0.03 25.73 < 2 ∗ 10−16
0.75 flow of calls ca
log(nP ort)lls 0.10 0.01 16.82 < 2 ∗ 10−16 itations we do
ca 10.80 < 2 ∗ 10−16 calls for new a
er
nPortNA 0.39 0.04
s tom Svc
LARGE 0.30 0.01 20.78 < 2 ∗ 10−16
23.06 < 2 ∗ 10−16 6. VALID
cu US
0.28
0.41
0.01
0.01 28.99 < 2 ∗ 10−16 It is importa
that results refl
Table 3: Number of calls regression. R2 = .36. of the data coll
We inspecte
process and int
24 / 28
Most predictors are statistically significance due to large sample curacy. Throu

33. Points that I liked about
the paper:
• Clear and suitable models constructed
• Emphasize on customerʼs perception of
a software
• Applicability to the real world
25 / 28

34. Points that I disliked:
• Evaluation of customer calls model
lacks insights
• Amount of effort needed to replicate the
study
• Terms are often misused and mixed
26 / 28

35. Audris Mockus
Empirical estimates of software availability of
deployed systems.
2006 IEEE International Symposium on Empirical Software Engineering
Audris Mockus, David Weiss
Interval quality: relating customer perceived
quality to process quality.
2008 International Conference on Software Engineering
Nachiappan Nagappan, Brendan Murphy, Victor Basili
The influence of organizational structure on
software quality: an empirical case study.
2008 International Conference on Software Engineering
27 / 28

36. 28 / 28

37. 28 / 28

38. 28 / 28

39. 28 / 28

40. 28 / 28

41. DISCUSSION
28 / 28