AW11
Session 
6/5/2013 3:45 PM 
 
 
 
 
 
 

"Agile and CMMI:
Friend or Foe?
A Lead Appraiser's View"
 
 
 

Presented by:
Ed Weller
Integrated Productivity Solutions, LLC
 
 
 
 
 
 
 

Brought to you by: 
 

 
 
340 Corporate Way, Suite 300, Orange Park, FL 32073 
888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Ed Weller
Integrated Productivity Solutions, LLC

Ed Weller is the principal in Integrated Productivity Solutions, providing solutions to companies
seeking to improve productivity. Ed is internationally recognized as an expert in software
engineering and in particular software measurement. His focus on quality started with his work
on the Apollo program with General Electric; was reinforced during his work as a hardware, test,
software, and systems engineer and manager on mainframe systems at Honeywell and Groupe
Bull; and continued as the process group manager on the Motorola Iridium Project. For the past
fourteen years, Ed has worked to improve the quality and productivity in small to very large
companies that develop engineering and IT applications.

 
Agile and CMMI: Friend of Foe?
A Lead Appraiser’s View

Ed Weller
Integrated Productivity Solutions
© 2013, E. Weller

Agenda
Cultural bias?
Misunderstandings?
Education
Application and examples

OR

2
Agenda
Cultural bias?
Misunderstandings?
Education
Appraisals

OR

3

My Biases
Large Systems
◦ Mainframe Hardware and OS
◦ Iridium Project

CMM®/CMMI® Lead Appraiser since 1994
Client organizations vary from 15 to
140,000 people
Strong proponent of software engineering
Pragmatic view
◦ “If it works, something was done right”
® CMM and CMMI are registered trademarks of Carnegie Mellon University
SM SCAMPI is a service mark of Carnegie Mellon University
4
Agilistas’ Biases or Misunderstanding?
CMMI means heavy documentation
CMMI means rigid control or constraining
processes
CMMI has no value

5

Agenda
Cultural bias?
Misunderstandings?
Education
Appraisals

OR

6
Demographics
How many of you
◦ Have a recognizable Life Cycle Process
g
y
Agile/Scrum?
Waterfall or other life-cycle?
ScrumBut?

◦
◦
◦
◦

Are project managers/Scrum Masters?
Are testers?
Are developers?
Are managers (one or more levels above project
managers/Scrum Masters)?
◦ Measurement analysts?
7

Is There CMMI Overhead?
Are you using the CMMI to improve or to
obtain a level rating via an appraisal?
◦ The CMMI can provide valuable insight to
activities or processes that can be useful
◦ The CMMI appraisal process – SCAMPISM
As with any evaluation, audit, or appraisal process
there will be overhead. The question is:
overhead
How much?
What is the value

8
Key Concepts of CMMI
Descriptive, not prescriptive
Not a process description - a model with
sets of practices grouped by process
areas – developed from experience
Four “groupings” of process areas
◦
◦
◦
◦

Project Management
Engineering
Support
Process Management

Version 1.3 acknowledged Agile
9

How Big Is Your Organization?
How many teams do you have in your
organization?
◦ One to thirty+ ??
◦ Do multiple teams contribute to software
releases?

As your organization gets larger, the
benefits from the Support and Process
Management Process Areas increase
Let’s briefly explore this theme …
10
High Level Mapping
CMMI Process Categories

Project Management
Engineering
Support
Process Management

Covered by

Scrum
Agile/XP
?
?

A lot more complex than suggested by the above,
but let’s look how CMMI can support Agile/Scrum
The intent is not to cover or explain the CMMI model
today. For more information and mapping, see the
works of Neil Potter, Hillel Glazer, Jeff Dalton, etc,
11

The “No Way” View

CMMI

Agile/Scrum

12
Another View – Some Overlap
CMMI and Agile/Scrum

CMMI

How much overlap do we have?

13

What Experience Is Showing
The degree of overlap “depends” – What
are your needs?

CMMI
Agile/Scrum

14
Why Is the Last Picture Accurate?
CMMI is a model
◦ CMMI is not a process or methodology
◦ Many ways to implement practices

Agile initially was a philosophy
◦ Moving towards an development method –
see Scott Ambler’s Disciplined Agile Delivery
◦ Scrum is a project management method

Both Agile and Scrum (referred to a Agile
hereafter) include a significant number of
CMMI Practices
15

Agenda
Cultural bias?
Misunderstandings?
Education
Appraisals

OR

16
Specifics at 50,000 Feet (1)
Project Management
◦ Project planning – do you
Establish requirements?
Agree on the work to be done in a sprint?
Base the work on past history?
Define your life cycle?
Estimate effort and cost?
Plan St k h ld involvement?
Pl Stakeholder i l
t?
Obtain commitment?

◦ Would you do any significant software
development without these activities ?
17

Specifics at 50,000 Feet (2)
Project Monitoring and Control – do you
◦
◦
◦
◦
◦

Monitor progress?
Monitor risks?
Conduct progress reviews?
Identify and analyze issues?
Manage corrective action?

Do you do most of these actions in the
daily standup? Or as a normal part of the
sprint?
18
A Few (More) Questions
How many of you have a set iteration or
sprint length?
What do you do on Day 1 of the
iteration?
Day 2…N
y
g
How many have a backlog?
How many estimate story points and
manage an iteration with planned
velocity?
19

Engineering/ Agile-XP (1)
AgileRequirements “Elicitation”
◦ Product Owner co-located
co located
◦ User Stories

Technical Solution
◦ Design
Pairing, Test Driven Design

◦ Coding
Pairing

Product Integration
◦ How do you get your code into production?
20
Engineering/ Agile-XP (2)
AgileTesting
◦ Testers part of team
◦ Test Driven Design
◦ CI, automated testing

Validation
◦ Demos at the end of sprints
◦ Working software rather than simulations,
analysis, or arm-waving

21

Is It Really That Straightforward?
Yes and No ☺
Yes because the methods developed over
Yes,
the last 10+ years have matured
◦ Do you know what you do in each day of your
sprint cycle?
◦ Do you have expectations for all the
stakeholders?

No, because there is actually more discipline
in functioning Agile teams than in many
waterfall projects, and discipline isn’t free
22
Support Process Areas (1)
This is a mixed bag
Some are a natural part of an Agile
development approach
◦ Configuration management – would anyone in
their right mind try to deliver code without a
CM system?
◦ Change control is built in to sprint
management (if done smartly)
◦ Consider Agile without Continuous
Integration (CI), and CI without tool support
23

Support Process Areas(2)
Measurement
◦ Velocity – don’t you need history of Velocity?
y
y
y
y
◦ Size (work) estimates – Fibonacci sequence or ?
◦ Defects (?) – many heated discussions about
“What is a defect”
Working backwards from what your users see is a good
way to start defining “defect”
y y
y
g
Ultimately, if you think you did it right and the test fails,
or production fails, there is a defect

◦ The larger the organization, the more outside
“pressure” on performance and skepticism (NIH)
regarding Agile (e.g., pairing)
24
Support Process Areas(3)
Process Assurance
◦ With 1 2 teams, probably a nuisance
1-2 teams
◦ But with 10-30 teams, how do you maintain
convergence of process (the way work gets
done)?
◦ How do you identify useful “deviations” from
norm,
the norm and propagate them across the
organization?
◦ Even with the best of intentions, objective
analysis can be lost
25

Support Process Areas(4)
Trade Studies (DAR)
◦ When there are multiple possible solutions,
how else would you decide?

The value of formal trade studies may be
at the organization or business level
◦ How many “big” decisions are made at the
team level – few and far between?
◦ How about method/process change based on
retrospectives?
◦ Tailoring or deviations?
26
Process Management(1)
This may be the sticky point – how much
process management do you need?
These process areas are intended to be
“organizational”
◦ How big is your organization – 1, 5, 10, 20,
20+ teams?
◦ How much should be relative to the number
of teams

27

Process Management(2)
Who is responsible for coordinating and
leading improvement efforts?
Who collects, analyzes, and communicates
metrics?
Who maintains the organization process
assets?
Responsibility and resources need to be
committed to these tasks
28
Organization Training
How do you organize training for new
tools technologies, and processes?
tools, technologies
How do you ensure mentoring effectively
conveys skills and knowledge?
Who identifies training needs?
g
,
g
Who organizes trainers, brown bag
lunches, etc?

29

What the Answers Mean
Every time your answer was “Yes, we do
that”, you have a process (“the way you
that
( the
get work done”)
The CMMI can provide a framework for
identifying needs and guidance for
implementation

30
Agenda
Cultural bias?
Misunderstandings?
Education
Appraisals

OR

31

How About Appraisals?
If your organization decides it needs or
wants a CMMI Level rating a “SCAMPI-A”
rating, SCAMPI-A
is required
◦ Not cheap or easy and there are built in costs
that do not scale down to small organizations
easily
◦ There are significant training costs
si nificant trainin c sts
Typically 5 days for each team member
Lead Appraiser cost

32
Appraisal Cost(1)
Significant misunderstanding of what
“documentation” is required
q
◦ In Agile, where are many of the “project
management” artifacts kept?
Wall charts/Visual Management boards
Velocity per iteration
Burn down
Test status
Etc

◦ All of these are “documentation” – the
difficulty is getting this information to the
appraisal team and showing a history of use
33

Appraisal Cost(2)
“A picture is worth a thousand words”
◦ Tool screenshots/tool access
◦ Pictures of wall charts

Appraisals of organizations using Agile can
rely on pictures as a primary source or
project documentation /history
There will be additional overhead if team
members and the Lead Appraiser are not
co-located with the teams being appraised
34
Summary
When used properly, the CMMI can be
used to effectively address areas of
software development and organizational
structure that neither Agile nor Scrum
handle
Appraising for a level is another story – it
will be costly and should be tied to
business needs

35

Agile and CMMI: Yes, They Can Work Together

  • 1.
        AW11 Session  6/5/2013 3:45 PM              "Agile and CMMI: Friendor Foe? A Lead Appraiser's View"       Presented by: Ed Weller Integrated Productivity Solutions, LLC               Brought to you by:        340 Corporate Way, Suite 300, Orange Park, FL 32073  888‐268‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2.
    Ed Weller Integrated ProductivitySolutions, LLC Ed Weller is the principal in Integrated Productivity Solutions, providing solutions to companies seeking to improve productivity. Ed is internationally recognized as an expert in software engineering and in particular software measurement. His focus on quality started with his work on the Apollo program with General Electric; was reinforced during his work as a hardware, test, software, and systems engineer and manager on mainframe systems at Honeywell and Groupe Bull; and continued as the process group manager on the Motorola Iridium Project. For the past fourteen years, Ed has worked to improve the quality and productivity in small to very large companies that develop engineering and IT applications.  
  • 3.
    Agile and CMMI:Friend of Foe? A Lead Appraiser’s View Ed Weller Integrated Productivity Solutions © 2013, E. Weller Agenda Cultural bias? Misunderstandings? Education Application and examples OR 2
  • 4.
    Agenda Cultural bias? Misunderstandings? Education Appraisals OR 3 My Biases LargeSystems ◦ Mainframe Hardware and OS ◦ Iridium Project CMM®/CMMI® Lead Appraiser since 1994 Client organizations vary from 15 to 140,000 people Strong proponent of software engineering Pragmatic view ◦ “If it works, something was done right” ® CMM and CMMI are registered trademarks of Carnegie Mellon University SM SCAMPI is a service mark of Carnegie Mellon University 4
  • 5.
    Agilistas’ Biases orMisunderstanding? CMMI means heavy documentation CMMI means rigid control or constraining processes CMMI has no value 5 Agenda Cultural bias? Misunderstandings? Education Appraisals OR 6
  • 6.
    Demographics How many ofyou ◦ Have a recognizable Life Cycle Process g y Agile/Scrum? Waterfall or other life-cycle? ScrumBut? ◦ ◦ ◦ ◦ Are project managers/Scrum Masters? Are testers? Are developers? Are managers (one or more levels above project managers/Scrum Masters)? ◦ Measurement analysts? 7 Is There CMMI Overhead? Are you using the CMMI to improve or to obtain a level rating via an appraisal? ◦ The CMMI can provide valuable insight to activities or processes that can be useful ◦ The CMMI appraisal process – SCAMPISM As with any evaluation, audit, or appraisal process there will be overhead. The question is: overhead How much? What is the value 8
  • 7.
    Key Concepts ofCMMI Descriptive, not prescriptive Not a process description - a model with sets of practices grouped by process areas – developed from experience Four “groupings” of process areas ◦ ◦ ◦ ◦ Project Management Engineering Support Process Management Version 1.3 acknowledged Agile 9 How Big Is Your Organization? How many teams do you have in your organization? ◦ One to thirty+ ?? ◦ Do multiple teams contribute to software releases? As your organization gets larger, the benefits from the Support and Process Management Process Areas increase Let’s briefly explore this theme … 10
  • 8.
    High Level Mapping CMMIProcess Categories Project Management Engineering Support Process Management Covered by Scrum Agile/XP ? ? A lot more complex than suggested by the above, but let’s look how CMMI can support Agile/Scrum The intent is not to cover or explain the CMMI model today. For more information and mapping, see the works of Neil Potter, Hillel Glazer, Jeff Dalton, etc, 11 The “No Way” View CMMI Agile/Scrum 12
  • 9.
    Another View –Some Overlap CMMI and Agile/Scrum CMMI How much overlap do we have? 13 What Experience Is Showing The degree of overlap “depends” – What are your needs? CMMI Agile/Scrum 14
  • 10.
    Why Is theLast Picture Accurate? CMMI is a model ◦ CMMI is not a process or methodology ◦ Many ways to implement practices Agile initially was a philosophy ◦ Moving towards an development method – see Scott Ambler’s Disciplined Agile Delivery ◦ Scrum is a project management method Both Agile and Scrum (referred to a Agile hereafter) include a significant number of CMMI Practices 15 Agenda Cultural bias? Misunderstandings? Education Appraisals OR 16
  • 11.
    Specifics at 50,000Feet (1) Project Management ◦ Project planning – do you Establish requirements? Agree on the work to be done in a sprint? Base the work on past history? Define your life cycle? Estimate effort and cost? Plan St k h ld involvement? Pl Stakeholder i l t? Obtain commitment? ◦ Would you do any significant software development without these activities ? 17 Specifics at 50,000 Feet (2) Project Monitoring and Control – do you ◦ ◦ ◦ ◦ ◦ Monitor progress? Monitor risks? Conduct progress reviews? Identify and analyze issues? Manage corrective action? Do you do most of these actions in the daily standup? Or as a normal part of the sprint? 18
  • 12.
    A Few (More)Questions How many of you have a set iteration or sprint length? What do you do on Day 1 of the iteration? Day 2…N y g How many have a backlog? How many estimate story points and manage an iteration with planned velocity? 19 Engineering/ Agile-XP (1) AgileRequirements “Elicitation” ◦ Product Owner co-located co located ◦ User Stories Technical Solution ◦ Design Pairing, Test Driven Design ◦ Coding Pairing Product Integration ◦ How do you get your code into production? 20
  • 13.
    Engineering/ Agile-XP (2) AgileTesting ◦Testers part of team ◦ Test Driven Design ◦ CI, automated testing Validation ◦ Demos at the end of sprints ◦ Working software rather than simulations, analysis, or arm-waving 21 Is It Really That Straightforward? Yes and No ☺ Yes because the methods developed over Yes, the last 10+ years have matured ◦ Do you know what you do in each day of your sprint cycle? ◦ Do you have expectations for all the stakeholders? No, because there is actually more discipline in functioning Agile teams than in many waterfall projects, and discipline isn’t free 22
  • 14.
    Support Process Areas(1) This is a mixed bag Some are a natural part of an Agile development approach ◦ Configuration management – would anyone in their right mind try to deliver code without a CM system? ◦ Change control is built in to sprint management (if done smartly) ◦ Consider Agile without Continuous Integration (CI), and CI without tool support 23 Support Process Areas(2) Measurement ◦ Velocity – don’t you need history of Velocity? y y y y ◦ Size (work) estimates – Fibonacci sequence or ? ◦ Defects (?) – many heated discussions about “What is a defect” Working backwards from what your users see is a good way to start defining “defect” y y y g Ultimately, if you think you did it right and the test fails, or production fails, there is a defect ◦ The larger the organization, the more outside “pressure” on performance and skepticism (NIH) regarding Agile (e.g., pairing) 24
  • 15.
    Support Process Areas(3) ProcessAssurance ◦ With 1 2 teams, probably a nuisance 1-2 teams ◦ But with 10-30 teams, how do you maintain convergence of process (the way work gets done)? ◦ How do you identify useful “deviations” from norm, the norm and propagate them across the organization? ◦ Even with the best of intentions, objective analysis can be lost 25 Support Process Areas(4) Trade Studies (DAR) ◦ When there are multiple possible solutions, how else would you decide? The value of formal trade studies may be at the organization or business level ◦ How many “big” decisions are made at the team level – few and far between? ◦ How about method/process change based on retrospectives? ◦ Tailoring or deviations? 26
  • 16.
    Process Management(1) This maybe the sticky point – how much process management do you need? These process areas are intended to be “organizational” ◦ How big is your organization – 1, 5, 10, 20, 20+ teams? ◦ How much should be relative to the number of teams 27 Process Management(2) Who is responsible for coordinating and leading improvement efforts? Who collects, analyzes, and communicates metrics? Who maintains the organization process assets? Responsibility and resources need to be committed to these tasks 28
  • 17.
    Organization Training How doyou organize training for new tools technologies, and processes? tools, technologies How do you ensure mentoring effectively conveys skills and knowledge? Who identifies training needs? g , g Who organizes trainers, brown bag lunches, etc? 29 What the Answers Mean Every time your answer was “Yes, we do that”, you have a process (“the way you that ( the get work done”) The CMMI can provide a framework for identifying needs and guidance for implementation 30
  • 18.
    Agenda Cultural bias? Misunderstandings? Education Appraisals OR 31 How AboutAppraisals? If your organization decides it needs or wants a CMMI Level rating a “SCAMPI-A” rating, SCAMPI-A is required ◦ Not cheap or easy and there are built in costs that do not scale down to small organizations easily ◦ There are significant training costs si nificant trainin c sts Typically 5 days for each team member Lead Appraiser cost 32
  • 19.
    Appraisal Cost(1) Significant misunderstandingof what “documentation” is required q ◦ In Agile, where are many of the “project management” artifacts kept? Wall charts/Visual Management boards Velocity per iteration Burn down Test status Etc ◦ All of these are “documentation” – the difficulty is getting this information to the appraisal team and showing a history of use 33 Appraisal Cost(2) “A picture is worth a thousand words” ◦ Tool screenshots/tool access ◦ Pictures of wall charts Appraisals of organizations using Agile can rely on pictures as a primary source or project documentation /history There will be additional overhead if team members and the Lead Appraiser are not co-located with the teams being appraised 34
  • 20.
    Summary When used properly,the CMMI can be used to effectively address areas of software development and organizational structure that neither Agile nor Scrum handle Appraising for a level is another story – it will be costly and should be tied to business needs 35