Iwsm2014 exponentially weighted moving average prediction in the software development process (eberhard kranich & thomas fehlmann)

Motivation
Warm Up: Shewhart Control Charts
Short-Run Control Charts
EWMA Q Control Charts
Appendix
Exponentially Weighted Moving Average
(EWMA) Prediction in the Software
Development Process
Thomas M. Fehlmann1 Eberhard Kranich2
1Euro Project Oce AG
Zurich, Switzerland
thomas.fehlmann@e-p-o.com
2Euro Project Oce
Duisburg, Germany
eberhard.kranich@e-p-o.com
T. Fehlmann, E. Kranich (Euro Project Oce) EWMA Prediction in the Software Development Process

Motivation
Appendix
Project Managment Task
Task
Design and implement a tool for monitoring and controlling the
software development process, especially its test phase.
Key Requirements
1 Visualization of concurrently running projects in one graphic.
2 Enable a graphical comparison of detected defect types.
3 Monitoring/Controlling asap after process/test phase start-up.
4 Implement a forecast functionality for controlling the process.

Motivation
Appendix
Historical Remarks
A Typical Control Chart
Subgroup Control Chart
Historical Remarks

Motivation
Appendix
Historical Remarks
X{ Control Chart
UCL
CL
LCL
X − Chart for Category: defects
day # defects
observations = 21
mean = 22.43
std.dev. = 6.5
UCL = 41.93
CL = 22.43
LCL = 2.93
beyond limits = 0
violating runs = 0
40
30
20
10
0 5 10 15 20

Motivation
Appendix
Historical Remarks
Category Control Chart
UCL
CL
LCL
X − Chart for Category: algorithm
day # algorithm
observations = 21
mean = 2.24
std.dev. = 3.48
UCL = 12.67
CL = 2.24
LCL = −8.19
beyond limits = 1
violating runs = 0
15
10
5
0
−5
0 5 10 15 20

Motivation
Appendix
Introduction
Q-Control Charts
Merging Q Control Charts
Handling Outliers
Why Short-Run Control Charts ?
1 A suciently large data set is not available to construct a
Shewhart control chart.
2 A process has to be monitored and controlled within a short
time after its start-up.
3 To stabilize an individual unit producing process as soon as
possible, the process must be analyzed automatically in
real-time.

Motivation
Appendix
Introduction
Q-Control Charts
Handling Outliers
References
Software Qual J (2013) 21:479–499
DOI 10.1007/s11219-012-9182-y Short-Run Control Charts for SDP
Monitoring the software development process using
a short-run control chart
Chih-Wei Chang • Lee-Ing Tong
Published online: 22 July 2012
Springer Science+Business Media, LLC 2012
Abstract Techniques for statistical process control (SPC), such as using a control chart,
have recently garnered considerable attention in the software industry. These techniques
are applied to manage a project quantitatively and meet established quality and process-performance
objectives. Although many studies have demonstrated the benefits of using a
control chart to monitor software development processes (SDPs), some controversy exists
regarding the suitability of employing conventional control charts to monitor SDPs. One
major problem is that conventional control charts require a large amount of data from a
homogeneous source of variation when constructing valid control limits. However, a large
dataset is typically unavailable for SDPs. Aggregating data from projects with similar
attributes to acquire the required number of observations may lead to wide control limits
due to mixed multiple common causes when applying a conventional control chart. To
overcome these problems, this study utilizes a Q chart for short-run manufacturing pro-cesses
as an alternative technique for monitoring SDPs. The Q chart, which has early
detection capability, real-time charting, and fixed control limits, allows software practi-tioners
to monitor process performance using a small amount of data in early SDP stages.
To assess the performance of the Q chart for monitoring SDPs, three examples are utilized
to demonstrate Q chart effectiveness. Some recommendations for practical use of Q charts
for SDPs are provided.
Keywords Software development process Statistical process control Control chart
Short production run Q chart
C.-W. Chang () L.-I. Tong
Department of Industrial Engineering and Management, National Chiao Tung University,
1001, Daxue Rd., Hsinchu City 300, Taiwan, ROC
e-mail: scott.iem95g@nctu.edu.tw; scott.nctu@gmail.com
L.-I. Tong
e-mail: litong@cc.nctu.edu.tw
123
Introducing Short-Run Control Charts for
Monitoring the Software Development Process
Thomas M. Fehlmann1), Eberhard Kranich2)
1)Euro Project Office, Zurich, Switzerland
2)T-Systems International GmbH, Telekom IT (PQIT), Bonn, Germany
1)Thomas.Fehlmann@e-p-o.com, 2)Eberhard.Kranich@t-online.de
Abstract
It is common practice in manufacturing industries that Statistical Process Control
(SPC) methods are applied for monitoring, controlling and improving processes
over time. One of the prominent SPC tools is the classical Shewhart control chart
which gives valuable insight into the sources of variation of a process, whereby
a variation is caused either by in-process interactions such as a man-machine in-teraction,
or by out-of-process events that deteriorate the process and thus make
the process unstable. Shewhart control charts are well suited for long run pro-cesses
so that actual in-process parameters or actual control limits can be es-tablished
on the basis of historical data gathered during a large number of the
process runs. Short run processes lack a sufficiently large set of historical data, so
that Shewhart control charts cannot be constructed. But short-run control charts,
also termed self-starting control charts, enable the monitoring and controlling of
a process within a short time after its start-up, when a large amount of historical
data are not at hand, and update the in-process parameters with each new process
run. Hence self-starting control charts such as Tukey’s control charts and Quesen-berry’s
Q-charts are highly appropriate for controlling and monitoring a software
development process and its various phases, for instance, when the software test
process is monitored by the faults-slip-through measurement process.
Keywords
Software Development Process (SDP), Statistical Process Control (SPC), short-run
control chart, self-starting control chart, Tukey’s control chart, Q-chart, faults-slip-
through measurement process
1 Introduction
The Six Sigma methodology with its DMAIC phase model is an accepted ap-proach
to improve an existing process systematically. In contrast, if an existing
process is to be redesigned completely or a new process is to design, the Design
for Six Sigma (DFSS) methodology with its mostly applied DMADV phase
MetriKon 2013

Motivation
Appendix
Introduction
Q-Control Charts
Handling Outliers
Speci

c References
SPC Q Charts for Start-Up Processes
and Short or Long Runs
CHARLES P. QUESENBERRY
North Carolina State University, Raleigh, North Carolina 27695-8203
Classical control charts are designed for processes where data to estimate the process parameters
and compute the control limits are available before a production run. For many processes, especially
in a job-shop setting, prod uction runs are not necessarily long and charting techniques are required
that do not depend upon knowing the process parameters in advance of the run. It is desirable to
begin charting at or very near the beginning of the run in these cases. We present here the needed
formulas so that charts for both the process mean and variance can be maintained from the start of
prod uction, whether or not prior information for estimating the parameters is available. These Q charts
are all plotted in a standardized normal scale, and therefore permit the plotting of different statistics
on the same chart. This will sometimes permit savings in the chart management program.
Introduction
THERE has recently been considerable interest in
II using SPC charting techniques in the job-shop environment.
Classical SPC charting methods such as
X, R, and S charts assume high volume manufacturing
processes where at least 25 or 30 calibration samples
of size 4 or 5 each can be gathered to estimate the
process parameters before on-line charting actually
begins. However, the job-shop environment involves
low-volume production and there is often a paucity
of relevant data available for estimating the process
parameters and establishing control limits prior to a
production run. In many applications there is no really
reliable data available for this purpose. Another important
practical problem in many job shops is that
there are so many different types of measurements
(i.e. part numbers) that a multitude of charts is required.
Thus standardized charts that permit different
statistics to be plotted on the same chart can be used
to simplify the chart management problem.
Specifically, we consider the following model setting.
Let
(1)
represent measurements that may be from a sequence
of consecutively produced parts. If the values Xr are
Dr. Quesenberry is a Professor of Statistics. He is a Senior
Member of ASQC.
Vol. 23, No. 3, July 1991 21 3
stochastically independent with the same distribution,
then this common distribution is called the process distribution
and its mean Il and variance 0-2 are called the
process mean and process variance. It should be kept in
mind that when the measurements are on consecutively
produced parts that the independence assumption
may be invalid due to autocorrelation, and that
the techniques given here are appropriate only for
independent observations. Methods of checking for
autocorrelation given in books such as Box and Jenkins
(1976) and Fuller (1976) can be used to detect
autocorrelation. The test of Durbin and Watson (1950,
1951, 1971) can be used to make a test for autocorrelation.
Marr and Quesenberry (1989) recently gave
a test for autocorrelation that can also be used in this
context and is particularly convenient for some types
of data encountered in applications in SPC. Also see
Montgomery and Mastrangelo (1991) in this issue.
Most presently available SPC charting methods such
as Shew hart charts, CUSUM charts, or geometric
moving average charts assume that data to estimate
the process parameters are available before the run
of parts giving the data in (1) is made. In effect, these
charting techniques assume that the values of the
process mean Il and process variance 0-2 are known
when the run of parts for (1) is begun. However, in
many cases the process mean and variance cannot be
known before the production run is begun, because
they change from run to run. This makes it difficult
to construct valid charts using presently available
Journal of Quality Technology

Motivation
Appendix
Introduction
Q-Control Charts
Handling Outliers
A Q Control Chart
UCL
LCL
2
0
−2
0 5 10 15 20
day
Q−Statistic
Defect Type
defects
Q Control Chart

Motivation
Appendix
Introduction
Q-Control Charts
Handling Outliers
UCL
LCL
2
0
−2
0 5 10 15 20
day
Q−Statistic
Defect Type
algorithm
Q Control Chart
UCL
LCL
4
2
0
−2
0 5 10 15 20
day
Q−Statistic
Defect Type
functions
algorithm
Q Control Chart
UCL
LCL
5.0
Statistic
2.5
Q−0.0
−2.5
day 0 5 10 15 20
Defect Type
functions
interface
algorithm
Q Control Chart
UCL
LCL
5.0
2.5
0.0
−2.5
0 5 10 15 20
day
Q−Statistic
Defect Type
defects
functions
interface
algorithm
Q Control Chart

Motivation
Appendix
Introduction
Q-Control Charts
Handling Outliers
Handling Outliers
UCL
LCL
2
0
−2
0 5 10 15 20
day
Q−Statistic
Defect Type
algorithm
Q Control Chart
UCL
LCL
2
0
−2
0 5 10 15 20
day
Q−Statistic
Defect Type
algorithm
Q Control Chart

Motivation
Appendix
An EWMA Q Control Chart
Fast Initial Response (FIR)
Forecasting
2
1
0
−1
−2
0 5 10 15 20
day
EWMA Q−Statistic
Defect Type
defects
EWMA Q Control Chart: l = 0.25, r = 2.998

Motivation
Appendix
Forecasting
2
1
0
−1
−2
0 5 10 15 20
day
EWMA Q−Statistic (fir)
Defect Type
defects

Motivation
Appendix
Forecasting
EWMA vs. FIR EWMA
2
1
0
−1
−2
0 5 10 15 20
day
EWMA Q−Statistic
Defect Type
defects
2
1
0
−1
−2
0 5 10 15 20
day
Defect Type
defects

Motivation
Appendix
Forecasting
EWMA vs. Modi

ed FIR EWMA
2
1
0
−1
−2
0 5 10 15 20
day
EWMA Q−Statistic
Defect Type
defects
2
1
0
−1
−2
0 5 10 15 20
day
EWMA Q−Statistic (mfir)
Defect Type
defects

Motivation
Appendix
Forecasting
FIR EWMA vs. Modi

ed FIR EWMA
2
1
0
−1
−2
0 5 10 15 20
day
Defect Type
defects
2
1
0
−1
−2
0 5 10 15 20
day
EWMA Q−Statistic (mfir)
Defect Type
defects

Motivation
Appendix
Forecasting
Q-Statistic Forecasting
One-Step Forecast Procedure
1 Utilize the R package forecast.
2 Apply the function ses to the actual Q-Statistics Qk(xk),
3 in order to obtain a forecast of Q-Statistic Qk+1(xk+1).
4 Check summary of ses for 80% and 90% prediction intervals.
5 If the forecast is not located in the intervals, then REACT,
6 else calculate the forecast xk+1 by the inverse Q-Statistic.

Motivation
Appendix
Forecasting
A Brief Example
Predicting Q18(x18)
fnct. forecast Q18(x18) 80% CI 95% CI
ses 0.348 2.078 [0:64; 1:33] [1:16; 1:85]
holt 0.182 2.078 [0:86; 1:22] [1:41; 1:77]
Predicting x18
fnct. forecast x18 x18 80% CI 95% CI
ses 24 36 [17; 31] [14; 35]
holt 23 36 [16; 30] [12; 35]

Motivation
Appendix
Standard Normal vs. Student's t Distribution
The Q-Statistic
The Q-Statistic: Updating Formulas
FIR/MFIR EWMA Calculations
Appendix

Motivation
Appendix
Standard Normal vs. Student's t Distribution
The Q-Statistic
The Q-Statistic: Updating Formulas
FIR/MFIR EWMA Calculations
N(0; 1) vs. t-Distribution
0.4
0.3
0.2
0.1
0.0
N(0, 1) vs. Student's t−distribution (df=1, 6, 10, 20)
−4 −2 0 2 4
density N(0, 1) t(df= 1) t(df= 6) t(df=10) t(df=20)

Iwsm2014 exponentially weighted moving average prediction in the software development process (eberhard kranich & thomas fehlmann)

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (20)

Similar to Iwsm2014 exponentially weighted moving average prediction in the software development process (eberhard kranich & thomas fehlmann)

Similar to Iwsm2014 exponentially weighted moving average prediction in the software development process (eberhard kranich & thomas fehlmann) (20)

More from Nesma

More from Nesma (20)

Recently uploaded

Recently uploaded (20)

Iwsm2014 exponentially weighted moving average prediction in the software development process (eberhard kranich & thomas fehlmann)