Understanding Software Delivery Delay

1. The Impact of Switching to a Rapid
Release cycle on Integration Delay of
Addressed Issues
Daniel A. Costa Shane McIntosh Uirá Kulesza Ahmed E. Hassan
UFRN McGill UFRN Queen’s

2. Issues can represent bug fixes,
enhancements, and new features
Bug fix
New feature
Enhancement
1

3. Software projects must attract and retain
users to survive
2

4. Slow delivery of fixed issues can drive
users away
So, when does this fix will get
added? Firefox release 22? 23?
(Issue report #883554)
3

5. Proponents of rapid release cycles argue
that fixed issues can be delivered faster
4

6. We study the rapid and traditional
releases of Mozilla Firefox
Traditional Rapid
VS
5

7. Traditional Rapid
111 releases 73 releases
We study the rapid and traditional
releases of Mozilla Firefox
VS
5

8. Traditional Rapid
111 releases 73 releases
34,673 issues 37,443 issues
We study the rapid and traditional
releases of Mozilla Firefox
VS
5

9. Traditional Rapid
111 releases 73 releases
34,673 issues 37,443 issues
We study the rapid and traditional
releases of Mozilla Firefox
VS
1.0 to 4.0 10 to 27
5

10. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
6

11. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1)
Comparing traditional
and rapid release
strategies
VS
6

12. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
3.6 3.6.1 3.6.2
VS
6

13. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
6

14. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
6

15. We measure the delivery delay
3.01.0 2.0
7

16. We measure the delivery delay
3.01.0 2.0
7

17. We measure the delivery delay
3.0
130 days
1.0 2.0
7

18. Issues are addressed more quickly in
rapid releases but are delivered more
slowly
500
50
5
1
Daystodeliverfixedissues
Traditional
Rapid
8

19. Issues are addressed more quickly in
rapid releases but are delivered more
slowly
500
50
5
1
Daystodeliverfixedissues
Traditional
Rapid
Data density
8

20. Issues are addressed more quickly in
rapid releases but are delivered more
slowly
500
50
5
1
Daystodeliverfixedissues
Traditional
Rapid
Median
Median
Data density
8

21. Issues are addressed more quickly in
rapid releases but are delivered more
slowly
500
50
5
1
Daystodeliverfixedissues
Traditional
Rapid
8

22. Fixed issues are delivered more
consistently in rapid releases
Traditional
Rapid
I can be
faster!
I have
consistent
gains!
9

23. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
10

24. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional releases
can deliver issues
more quickly
10

25. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional releases
can deliver issues
more quickly
10

26. Minor releases are how traditional
releases deliver issues more quickly
500
50
5
1
Major
traditional
Minor
traditional
Daystodeliverfixedissues
11

27. Minor releases are how traditional
releases deliver issues more quickly
500
50
5
1
Major
traditional
Minor
traditional
Daystodeliverfixedissues
Major
traditional
Rapid
11

28. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional releases
can deliver issues
more quickly
12

29. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional minor
releases lead to faster
delivery of issues
Traditional releases
can deliver issues
more quickly
12

30. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid release
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional minor
releases lead to faster
delivery of issues
Traditional releases
can deliver issues
more quickly
12

31. Factors that can cause delivery delay
Number of Impacted
Files
13

32. Factors that can cause delivery delay
Number of Impacted
Files
I Think this bug
is due to…
Number of comments
13

33. Factors that can cause delivery delay
Number of Impacted
Files
Number of comments Experience of the
resolvers
VS
I Think this bug
is due to…
13

34. Factors that can cause delivery delay
Number of Impacted
Files
Number of comments Experience of the
resolvers
VS
And more
I Think this bug
is due to…
13

35. A fixed issue is delayed if it is not
shipped in the next upcoming release
3.01.0 2.0
14

36. A fixed issue is delayed if it is not
shipped in the next upcoming release
3.01.0 2.0Not delayed
14

37. A fixed issue is delayed if it is not
shipped in the next upcoming release
3.01.0 2.0 Delayed
14

38. We build logistic regression models to
study factors associated with delay
Trad.
Rapid
15

39. We build logistic regression models to
study factors associated with delay
Regression models
Trad.
Learn from
Rapid
Learn from
15

40. We build logistic regression models to
study factors associated with delay
Regression models
not delayed
Trad.
Learn from Answer
delayed
not delayed
Rapid
Learn from Answer
delayed
15

41. We build logistic regression models to
study factors associated with delay
Regression models
not delayed
Trad.
Learn from Answer
delayed
not delayed
Rapid
Learn from Answer
delayed
VS
15

42. 3.02.0
Traditional
Prioritization is different along strategies
16

43. 3.02.0
Traditional
Prioritization is different along strategies
Backlog of issues
16

44. 3.02.0
Traditional
Prioritization is different along strategies
16

45. 3.02.0
Position 4 in the backlog
Traditional
Prioritization is different along strategies
16

46. 3.02.0
Position 4 in the backlog
Traditional
Prioritization is different along strategies
0.95
0.90
0.85
0.80
0 5,000 10,000
Position in the backlog
Probabilityofdelay 16

47. 3.02.0
Position 4 in the backlog
Traditional
2.0
Rapid
3.0
Prioritization is different along strategies
0.95
0.90
0.85
0.80
0 5,000 10,000
Position in the backlog
Probabilityofdelay 16

48. 3.02.0
Position 4 in the backlog
Traditional
2.0
Rapid
3.0
Prioritization is different along strategies
0.95
0.90
0.85
0.80
0 5,000 10,000
Position in the backlog
Probabilityofdelay
Release cycle backlog
16

49. 3.02.0
Position 4 in the backlog
Traditional
2.0
Rapid
3.0
Prioritization is different along strategies
0.95
0.90
0.85
0.80
0 5,000 10,000
Position in the backlog
Probabilityofdelay 16

50. 3.02.0
Position 4 in the backlog
Traditional
2.0
Rapid
3.0
Prioritization is different along strategies
0.95
0.90
0.85
0.80
0 5,000 10,000
Position in the backlog
Probabilityofdelay
Position 2 in the cycle backlog
16

51. 3.02.0
Position 4 in the backlog
Traditional
2.0
Rapid
3.0
Prioritization is different along strategies
0.95
0.90
0.85
0.80
0 5,000 10,000
Position in the backlog
ProbabilityofdelayProbabilityofdelay
0.95
0.90
0.85
0.80
Position in the cycle backlog
0 1,000 3,000
Position 2 in the cycle backlog
16

52. Traditional releases behave as a queue
while rapid releases behave as a stack
VS
17

53. 1.0 2.0 3.0
VS
Traditional releases behave as a queue
while rapid releases behave as a stack
17

54. Released Completed
1.0 2.0 3.0
VS
Traditional releases behave as a queue
while rapid releases behave as a stack
17

55. Released Completed
1.0 2.0 3.0
3.0
2.0
1.0
VS
Traditional releases behave as a queue
while rapid releases behave as a stack
17

56. Released Completed
1.0 2.0 3.0
3.0
2.0
1.0
VS
Traditional releases behave as a queue
while rapid releases behave as a stack
CompletedReleased
17

57. Are addressed issues delivered more
quickly in rapid releases?
Case study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid releasing
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional minor
releases lead to faster
delivery of issues
Traditional releases
can deliver issues
more quickly
1

58. Are addressed issues delivered more
quickly in rapid releases?
Case study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid releasing
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional minor
releases lead to faster
delivery of issues
Traditional releases
can deliver issues
more quickly
Traditional and rapid
releases prioritize
issues differently
1

59. In Summary

60. Are addressed issues delivered more
quickly in rapid releases?
Empirical study structure
(RQ.1) (RQ.2) (RQ.3)
Comparing traditional
and rapid releasing
strategies
Studying minor and
major releases
Studying delay factors
of each release
strategy
3.6 3.6.1 3.6.2
VS
Traditional minor
releases lead to faster
delivery of issues
Traditional releases
can deliver issues
more quickly
Traditional and rapid
releases prioritize
issues differently

62. Fixed issues are delivered more
consistently in rapid releases
Traditional
Rapid
I can be
faster!
I have
consistent
gains!

64. Minor releases are how traditional
releases deliver issues more quickly
500
50
5
1
Major
traditional
Minor
traditional
Daystodeliverfixedissues
Major
traditional
Rapid

66. 3.02.0
Position 4 in the backlog
Traditional
2.0
Rapid
3.0
Prioritization is different along strategies
0.95
0.90
0.85
0.80
0 5,000 10,000
Position in the backlog
ProbabilityofdelayProbabilityofdelay
0.95
0.90
0.85
0.80
Position in the cycle backlog
0 1,000 3,000
Position 2 in the cycle backlog