1
2
52 weeks release cycle time               6 weeks release cycle time
                     30

                     25
Number of Releases




                     20

                     15

                     10

                     5

                     0
                          2004   2005   2006   2007   2008     2009   2010   2011   2012    2013
                                                             Year
                                                                                                   3
4
Speed   Quality


                  5
Case Study



• 20 Alpha         • 5 Alpha
• 20 Beta          • 5 Beta
• 23 Minor         • 6 Minor
• 2 Major          • 5 Major




                               6
 Median Daily Crash Count
        (lower is better)

    Median Uptime
      (higher is better)
                             7
13                      20
Number of Post Release Bugs /




                                5

                                4

                                3
           Day




                                2

                                1

                                0
                                    Traditional Release (TR)   Rapid Release (RR)



                                                                                    8
250K
Median Daily Crash Count




                           200K


                           150K


                           100K


                           50K


                            0K
                                  Traditional Release (TR)   Rapid Release (RR)



                                                                                  9
1200
                           1080
                            960
Median Uptime in Seconds




                            840
                            720
                            600
                            480
                            360
                            240
                            120
                              0
                                  Traditional Release (TR)   Rapid Release (RR)


                                                                                  10
YES… BUT
More investigations are needed…

                                  11
 Fixing time of bugs
     (lower is better)



                         12
100%
                   90%
                   80%
                   70%
% of Bugs Fixed




                   60%
                   50%
                   40%
                   30%
                   20%
                   10%
                    0%
                         Traditional Release   Rapid Release (RR) -   Traditional Release   Rapid Release (RR) -
                             (TR) - Main              Main                (TR) - Beta              Beta


                                  Main Releases                                   Beta Releases            13
654                    159
                  100
                   90
                   80
                   70
Bug Age in Days




                   60
                   50
                   40
                   30
                   20
                   10
                    0
                        Traditional Release (TR)   Rapid Release (RR)


                                                                        14
15
16
4.0      5.0     6.0       7.0     8.0   9.0
            100%
                           90%                                                       Version8

                           80%
                                                                                     Version7
% of Stale Crash Reports




                           70%
                           60%                                                       Version6

                           50%
                                                                                     Version5
                           40%
                           30%                                                       Version4

                           20%
                                                                                     Version3.6
                           10%
                           0%
                                 012345678901234567012345678012345601234501234
                                                    Weeks since Last Release


                            Approximately 20% of users remain on a stalled version
                                                                                           17
18
19
20

Do Faster Releases Improve Software Quality?

  • 1.
  • 2.
  • 3.
    52 weeks releasecycle time 6 weeks release cycle time 30 25 Number of Releases 20 15 10 5 0 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Year 3
  • 4.
  • 5.
    Speed Quality 5
  • 6.
    Case Study • 20Alpha • 5 Alpha • 20 Beta • 5 Beta • 23 Minor • 6 Minor • 2 Major • 5 Major 6
  • 7.
     Median DailyCrash Count (lower is better)  Median Uptime (higher is better) 7
  • 8.
    13 20 Number of Post Release Bugs / 5 4 3 Day 2 1 0 Traditional Release (TR) Rapid Release (RR) 8
  • 9.
    250K Median Daily CrashCount 200K 150K 100K 50K 0K Traditional Release (TR) Rapid Release (RR) 9
  • 10.
    1200 1080 960 Median Uptime in Seconds 840 720 600 480 360 240 120 0 Traditional Release (TR) Rapid Release (RR) 10
  • 11.
  • 12.
     Fixing timeof bugs (lower is better) 12
  • 13.
    100% 90% 80% 70% % of Bugs Fixed 60% 50% 40% 30% 20% 10% 0% Traditional Release Rapid Release (RR) - Traditional Release Rapid Release (RR) - (TR) - Main Main (TR) - Beta Beta Main Releases Beta Releases 13
  • 14.
    654 159 100 90 80 70 Bug Age in Days 60 50 40 30 20 10 0 Traditional Release (TR) Rapid Release (RR) 14
  • 15.
  • 16.
  • 17.
    4.0 5.0 6.0 7.0 8.0 9.0 100% 90% Version8 80% Version7 % of Stale Crash Reports 70% 60% Version6 50% Version5 40% 30% Version4 20% Version3.6 10% 0% 012345678901234567012345678012345601234501234 Weeks since Last Release Approximately 20% of users remain on a stalled version 17
  • 18.
  • 19.
  • 20.