Accelerate your Agile success with in-depth research and smarter decisions. Michael Mah of QSM Associates shows you what it takes to find and utilize patterns of successful Agile development in this quarterly XBOSoft webinar.

1. When Agile Is a
Quality Game Changer:
Predictive Forecasting of Defect Profiles
with QSM Associates’ Michael Mah
#AgileGameChanger
XBOSoft, Inc. All Rights Reserved.

2. About XBOSoft
Dedicated to Software Quality Improvement
Founded in 2006
We speed products to market with our expert:
• Software QA consulting
• Software testing
Global team with offices in San Francisco & Beijing
“Thorough, accurate and fast”
XBOSoft, Inc. All Rights Reserved.

3. House Rules
 Participants other than the speakers are muted
 Ask questions in the GoToWebinar control on the
right side of your screen or through Twitter
@XBOSoft
 Questions may be asked throughout the webinar -
we’ll try to answer them at the end
 You’ll receive info on recording after the webinar
Webinar Hashtag: #AgileGameChanger
XBOSoft, Inc. All Rights Reserved.

4. Meet the XBOSoft Team
Philip Lew
CEO and Founder, XBOSoft
•Relevant specialties and
passions
o Software quality process,
evaluation, measurement
and improvement
o Software quality in use /
UX design
o Mobile User Experience
and usability
o Cycling and travel
XBOSoft, Inc. All Rights Reserved.
Rachel Geibig
Marketing Manager, XBOSoft
•Emails you about our
industry hot topic
webinars
•Invites you to download
our latest trends in
software testing white
papers

5. With over 25 years of experience, Michael and his partners have
derived productivity patterns for thousands of software projects
collected worldwide across engineering and business applications.
His current work examines time-pressure dynamics of teams, and
its role in project success and failure. In addition to his background
in physics and electrical engineering, he is a mediator specializing
in dispute resolution for technology projects.
• Director of the Benchmarking Practice at the Cutter Consortium, a Boston-based IT think-tank
• Past editor of IT Metrics Strategies publication
• Managing partner at QSM Associates Inc. based in Massachusetts
• Teaches, writes, and consults to companies on measuring, estimating and managing software
projects, whether in-house, offshore, waterfall, or agile
He has a degree in electrics engineering from Tufts University. His training on dispute resolution,
mediation, and participatory processes is from the Program on Negotiation at Harvard Law School and
the Radcliffe Institute for Advanced Study. Michael is also a private pilot and lives in the mountains of
Western Massachusetts.
Michael Mah

6. (#6)

9. (#9)

10. Rayleigh Curve Defect Rate
0
20
40
60
80
100
120
140
160
Defect Type (All)
Count of Severity*
Create Date Status Mode Status Severity* TR-Version
Product+*

11. (#11)

12. (#12)

13. (#13)

14. (#14)

15. (#15)

16. The QSM SLIM Database
QSM maintains the world’s largest benchmarking database of
12,000+ completed software projects collected worldwide. We
put industry productivity statistics on the desktop.
The QSM SLIM database contains projects in all industries,
waterfall, Agile, offshore/outsourced, in-house, new
development, and maintenance.
SLIM tools enable managers to measure and estimate and in-
flight forecast Agile and waterfall projects.

17. Partial List of Clients
British Telecom
SAP
Microsoft
Intel
AT&T/BellSouth
Nationwide
Motorola
Verizon Wireless
Roche Diagnostics
Fiserv Corp
IBM Global
Misys Healthcare
JPMorganChase
Boeing
Bank of New York
Mellon
Lockheed Martin
Progressive Insurance
DirecTV

18. (#18)

23. Input to SLIM
Size
Time
Defects
Effort

24. Trendline Assessment – Defects/Quality
Defects During Test
100 1,000
Effective SLOC (thousands)
10
100
1,000
10,000
Errors(SysInt-Del)
Rel 5.0
Rel 6.0
Rel 6.5
Rel 7.5
Rel 7.0
Rel 8.0
Rel 5.0
Rel 6.0
Rel 6.5
Rel 7.5
Rel 7.0
Rel 8.0
Business Sy stems Av ionic Sy stems Command & Control Microcode Sy stems Process Control QSM 2005 Business
Av g. Line Sty le 1 Sigma Line Sty le
Far Fewer Defects: 50% - 66% Below Industry

25. Trendline Assessment – Build Phase Schedule
Main Build Phase Duration vs Size
100 1,000
Effective SLOC (thousands)
1
10
100
Time(Months)
Rel 5.0
Rel 6.0
Rel 6.5
Rel 7.5
Rel 7.0
Rel 8.0
Rel 5.0
Rel 6.0
Rel 6.5
Rel 7.5
Rel 7.0
Rel 8.0
Business Sy stems Av ionic Sy stems Command & Control Microcode Sy stems Process Control QSM 2005 Business
Av g. Line Sty le 1 Sigma Line Sty le
Schedules are Half Industry

26.
Industry
Average
Current
Performance
Delta
Project Cost $3.5 Million $2.2 Million -$1.3M
Schedule 12.6 months 7.8 months -4.8 mos
QA Defects 242 121 -50%
Staffing 35 35 n/a
Follett vs. Industry Average

27. Short Feedback Loops
Paired programmers
Instantaneous code reviews
Accelerated learning and
execution
Face to face communication
channel

28. Time Boxing
Short iterations
Clear and discernible
progress
Anticipation of the next
important feature
Efficiency

29. Avoiding Burnout
XP = Sustainable pace
40 Hour Work Weeks
Prevent productivity
collapse for
overworked teams

30. Transparency
“Transparency is a great
floodlight. People who
thrive in political
maneuvering hate
SCRUM…”
- Ken Schwaber

31. High-Bandwidth Communication
The best teams have
“wide-open pipes”
Domain knowledge moves
among the team
Information flows rapidly
and accurately

32. Avoiding Waste and Costly Rework
Rework has high cost
Rework takes time
Rework creates defects
Rework is bad
Refactoring can be a cover up

33. Norbert Wiener, 1894-1964

34. “He noticed that the Feedback principle is a key
feature of life forms from the simplest plants to
the most complex animals, which change their
reactions in response to their environment.”
Keywords: Shannon’s Information Theory, Cybernetics
Norbert Wiener, 1894-1964

35. Example Outsourced Project – Whiteboard Sketch

36. Input to SLIM
Size
Time Effort

37. SLIM Replica – Release 9.5
Staffing & Probability Analysis
Avg Staff (people)
<Single Goal - MBI 4.433>
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Jan
'06
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan
'07
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
0
20
40
60
80
100
AvgStaff(people)
876543210
CD
R&D
C&T
Milestones
0 - CSR
1 - SRR
2 - HLDR
3 - LLDR
4 - CUT
5 - IC
6 - STC
7 - UAT
8 - FCR
9 - 99R
10 - 99.9R
Milestones
0 - CSR
1 - SRR
2 - HLDR
3 - LLDR
4 - CUT
5 - IC
6 - STC
7 - UAT
8 - FCR
9 - 99R
10 - 99.9R
SOLUTION PANEL <Single Goal - MBI 4.433>
Duration
Effort
Cost
Peak Staff
MTTD
Start Date
C&T
14.5
716
4153.2
74.8
0.1
8/4/2006
Life Cycle
19.6
919
5328.5
74.8
0.1
3/1/2006
Months
PM
$ (K)
people
Days
PI=21.3 MBI=4.4 Eff SLOC=686,000
CONTROL PANEL <Single Goal - MBI 4.433>
PI
17.0 25.5
21.3
Peak Staff
59.8 89.7
74.8
Eff SLOC (K)
549 823
686

38. Main Build Phase Schedule vs. Size
1 10 100 1,000 10,000
Effective SLOC (thousands)
1
10
100
1,000
DC&TDuration(Months)
AB Rel 1.0
AB Rel 9.5
AB Rel 9.0
AB Rel 1.0
AB Rel 9.5
AB Rel 9.0
AB Medical All AB Medical Sy stems Special Project Misy s - All Projects Av g. Line Sty le 1 Sigma Line Sty le
Trendline Assessment – Main Build Schedule
Company – All Projects
Deadline target for Rel 9.0 is aggressive but consistent
within the band of history. Rel 9.5 and Rel 10
scope/dates are off the chart.

39. Main Build Phase Staff vs Size
1 10 100 1,000 10,000
Effective SLOC (thousands)
1
10
100
1,000
DC&TPeakStaff(People)
AB Rel 9.5
AB Rel 10
AB Rel 9.0
AB Rel 9.5
AB Rel 10
AB Rel 9.0
AB Medical All AB Medical Sy stems Special Project Misy s - All Projects Av g. Line Sty le 1 Sigma Line Sty le
Planned staffing profiles (RED projects)
are much more aggressive than history.
Tight deadlines and time-to-market
are the drivers.
Trendline Assessment – Main Build Staffing
Company – All Projects

40. Trendline Assessment – Defects/Quality
Defects During Test vs Size
1 10 100 1,000 10,000
Effective SLOC (thousands)
0.001
0.01
0.1
1
10
100
Errors(SysInt-Del)(thousands)
AB Medical All AB Medical Sy stems Special Project Misy s - All Projects Av g. Line Sty le 1 Sigma Line Sty le
Ambitious scope, aggressive staffing, and tight deadlines
result in (higher/lower) defects. (Fill in answer)
[WATCH YOUR WARRANTEES]
Company – All Projects

41. Defects Found
6 12 18 24 30 36 42 48 54 60 66 72 78 84 90
05/17
'03
06/28 08/09 09/20 11/01 12/13 01/24
'04
03/06 04/17 05/29 07/10 08/21 10/02 11/13 12/25 02/05
'05
0
50
100
150
200
250
300
350
Defects
10987654310
1098765
Current Plan Actuals Green Control Bound Yellow Control Bound Project: CEP
Defects vs. Industry Average

42. Defects Found
6 12 18 24 30 36 42 48 54 60 66 72 78 84 90
05/17
'03
06/28 08/09 09/20 11/01 12/13 01/24
'04
03/06 04/17 05/29 07/10 08/21 10/02 11/13 12/25 02/05
'05
0
100
200
300
400
500
Defects
10987654310
1098765
Current Plan Actuals Green Control Bound Yellow Control Bound Project: CEP
Scaled Up Defect Curve – Outsourced Project

44. Nov 19
Forecast to Complete (4 Month Delay)

45.
Industry
Average
Offshore
Average
Delta
Project Cost $3.5 Million $3.2 Million -$0.3M
Schedule 12.6 months 9.6 months -3.0 mos
QA Defects 242 677 +280%
Staffing 35 50 +15
* Normalized to same project size
Offshore vs. Industry Average

46.
Industry
Average
Current
Performance
Delta
Project Cost $3.5 Million $2.2 Million -$1.3M
Schedule 12.6 months 7.8 months -4.8 mos
QA Defects 242 121 -50%
Staffing 35 35 n/a
Follett vs. Industry Average

47. The force of concentration, or “clustering” of
human creativity and talent… the powerful
(economic) gains that come when smart and
talented people (co-)locate in close proximity to
one another…
Robert Lucas Revisited

48. Some
Practical
Advice

49. The Source of Our Troubles

50. Take a Little More Time

51. Build a Little Less

52. Use Small A-Teams

53. Give Them the Best Tools

57. (#60)

58. (#61)

60. Curve Fitting and Creating Agile Trends
Agile Trends - Iterations/Build Phase
Time-to-Market
10 100 1,000
New + Modified Size (thousands)
1
10
100
Months
Effort
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Person-Months
Average Staff
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
Headcount
Bugs During QA
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Defects
All Systems Avg. Line Style 1 Sigma Line Style

61. New Agile Benchmark Trends
Agile Trends - Iterations/Build Phase
Time-to-Market
10 100 1,000
New + Modified Size (thousands)
1
10
100
Months
Effort
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Person-Months
Average Staff
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000 Headcount
Bugs During QA
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Defects

62. The
SLIM
-Suite
SLIM-Estimate:
Size, Schedule,
Cost & Quality
Estimating
SLIM-DataManager
Software Project Metrics Repository
SLIM-Control:
Variance
Analysis
&
Adaptive
Forecasting
SLIM-Metrics:
Industry
Benchmarking &
Process
Improvement
SLIM-
MasterPlan:
Incremental
Development &
Project
Aggregation
TheSLIM-Suite
Software Lifecycle Management

63. (#66)

64. QSM SLIM Agile Benchmark Trends
Agile Trends - Iterations/Build Phase
Time-to-Market
10 100 1,000
New + Modified Size (thousands)
1
10
100
Months
Effort
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Person-Months
Average Staff
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
Headcount Bugs During QA
10 100 1,000
New + Modified Size (thousands)
1
10
100
1,000
10,000
Defects

65. (#68)

66. Contact Michael
Michael Mah
Managing Partner
QSM Associates, Inc.
Pittsfield MA USA
email: michael.mah@qsma.com
website: www.qsma.com
twitter: @michaelcmah
tel: 1 413-499-0988
Michael Mah
Managing Partner
QSM Associates, Inc.
Pittsfield MA USA
email: michael.mah@qsma.com
website: www.qsma.com
twitter: @michaelcmah
tel: 1 413-499-0988

67. Post your webinar questions on Twitter @XBOSoft
Registrants will receive an email with information on where to view the recording
and slides from today’s webinar.
Join us to keep updated on all our webinars, reports and white papers:
facebook.com/xbosoft
+xbosoft
linkedin.com/company/xbosoft
Check out our blog: http://xbosoft.com/software-quality-blog/
Download our free white papers: http://xbosoft.com/knowledge-center/
Email us with ideas for future webinars or questions regarding our services!
services@xbosoft.com
Thank you!
Q&A
www.xbosoft.com
XBOSoft, Inc. All Rights Reserved.