Keynote discussing how agile and CMMI can work together, the challenges each one has and how agile organisations move towards CMMI Level 5.

1. Portugal
Agile Methods and Capability Maturity:
Addressing the Paradox
Professor Ian Allison
Head of School of Computing Science and Digital Media
Robert Gordon University, Aberdeen, UK
2013-10-18

2. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Introduction
2

3. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Introduction
3

4. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Introduction
4

5. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Introduction
5

6. Key Themes
© 2013 CMMI Portugal Conference Series – All Rights reserved.
• Background
• Agile – the maturity paradox?
• Case study of a high maturity organisation
• Good practices to be considered
6

7. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Agile Development
7

8. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Agile Processes
8

9. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Current Status of Agile
84% of organisations using agile
50% started in the last 2 years
35% of companies had distributed agile projects
Main Methodology: SCRUM [54%& 11% SCRUM/XP hybrid]
Key benefits:
• Manage change
• Improve productivity
• Improve project visibility
• Enhanced quality of software
VersionOne, 2013
9

10. Key techniques
Coding Standards
© 2013 CMMI Portugal Conference Series – All Rights reserved.
Velocity
Burndown/team estimation
Release Planning
Retrospectives
Unit testing
Iteration Planning
Daily Standup
0
20
40
60
80
100
VersionOne, 2013 10

11. © 2013 CMMI Portugal Conference Series – All Rights reserved.
The Problem With Agile
11

12. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Leading Causes of Failure
•
•
•
•
•
Culture at odds with agile
12%
External pressure to follow waterfall
11%
Broader organisational communication problem 11%
Lack of experience with agile
9%
Lack of management support
6%
(18% claimed no failed projects)
VersionOne, 2013
12

13. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Main Concerns with Agile
•
•
•
•
•
•
•
•
Lack of upfront planning
Loss of management control
Management opposition
Lack of documentation
Lack of predictability
Lack of engineering discipline
Inability to scale
Regulatory compliance
VersionOne, 2013
13

14. How Agile Addresses the CMMI Practices in
3 Case Studies
© 2013 CMMI Portugal Conference Series – All Rights reserved.
CMMI Goal
Agile Practices
Requirement
management
Stories; product & sprint backlog; sprint planning; sprint reviews; planning
games; information radiator; Daily meetings; on-site customer; selforganising teams; planning days; release days;
Estimates
Sprint planning; product backlog; Sprint backlog; Planning games; tasks and
effort estimations 1-2 weeks ahead; information radiator; planning days
Project Planning
Planning games; task on information radiator; product backlog
Commitment to
plan
Planning games; self-organising teams; on-site customer; reflection
workshops; planning days; task cards on information radiator; release days
Project tracking
Short planning cycles; Burn down charts; project velocity; visual control;
daily meetings;
Configuration
Management
Collective ownership; small releases; continuous integration;
14
Paulk, 2001 & Pikkarainen & Mantyniemi, 2006

15. Specific Goals – Spanish Experience
12
© 2013 CMMI Portugal Conference Series – All Rights reserved.
10
8
6
4
2
0
Garzas & Paulk, 2013
15

16. © 2013 CMMI Portugal Conference Series – All Rights reserved.
How Agile Companies Move Towards CMMI
Level 5
DTE Energy (Baker, 2005)
• Important role for SEPG
• Create practice groups: Project Management; Core
development; Process Management & Measurement
• PMO; Information office
• Education & Socialisation of the idea of CMMI 5
• Communication – messages to key audiences
Pragmatics (Cohen & Glazer, 2009)
- Metrics programme that adds value
16

17. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Key Challenges for High Maturity
•
•
•
•
•
•
Need for recorded evidence
Need for project planning and control
Self-Organizing vs Planned
Organisational level of standardisation of process
Project Metrics and performance prediction
Organisational level data base-lining
17

18. © 2013 CMMI Portugal Conference Series – All Rights reserved.
High Maturity Case Study
• A CMMI level 5 Indian organisation working onshore &
offshore development
• Large enterprise with a revenue of $450m
• Study over the last year included site visits that allowed
interviews with 38 practitioners, observations and a review
of confidential agile process documents
18

19. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Agile History
• Started using in 2002… ad hoc use…
“so small projects here and small projects there, but since
2009 we have seen a huge uptake of agile …some
[projects] because the customer says; some because
there were a lot of problems so we recommended [agile]”
(Corporate Lead Architect)
• Then…
“the size of agile projects really increased. So instead of
having 20 or 10 members on projects, now suddenly we
have 140, you know large offshore development
programs being run in agile” (Corporate Lead Architect)
• Now 600 people … 20% of workforce on agile projects
19

20. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Recorded Evidence
• Records required in CMMI for:
• Decision rationale
• Project review and evaluation
• It is possible to have evidence in many other forms…
20

21. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Recorded Evidence
• Records required in CMMI for:
• Decision rationale
• Project review and evaluation
• It is possible to have evidence in many other forms…
21

22. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Recorded Evidence – Case Study
Practice Head stated that “Sufficient evidence is required so
that “Senior Management Review and Risk Management can
be performed [so] there is no contradiction with agile”
Evidence is typically recorded on agile projects often takes
the form of:
• source code annotations (for lowest level decisions)
• email discussions
• Intranet project content management systems
• Wiki information repositories that organically evolve over
the project lifetime
22

23. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Project Planning and Control
In CMMI it is important to have defined schedules, built from
defined requirements, upfront design and known productivity
rates
In agile approaches, evolution and responding to change
takes precedence over upfront planning.
However, the project plan that is expected in CMMI need not
be “what” but can focus on “how”.
23

24. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Project Planning and Control
In CMMI it is important to have defined schedules, built from
defined requirements, upfront design and known productivity
rates
In agile approaches, evolution and responding to change
takes precedence over upfront planning.
However, the project plan that is expected in CMMI need not
be “what” but can focus on “how”.
24

25. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Project Planning and Control
In CMMI it is important to have defined schedules, built from
defined requirements, upfront design and known productivity
rates
In agile approaches, evolution and responding to change
takes precedence over upfront planning.
However, the project plan that is expected in CMMI need not
be “what” but can focus on “how”.
25

26. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Project Planning and Control – Case Study
It was clear to the organisation that “every project needs to
have a project goal and a defined process … so, preparing
the initial project plan is not a problem” (Practice Head –
SEPG).
Partly this was because even in agile …
“in a distributed environment involving large team size, [then]
some form of upfront design is essential” (RK – Agile Coach,
Feb. 2013)
26

27. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Self-organising vs Planned
In CMMI:
• a consistent set of processes an organization is believed
to be able to reduce the variability between projects
• rigorous project management and control through a known
set of software engineering activities is the way this is
achieved.
In Agile:
• the emphasis is on self-organization and emergence
• product features emerge as the development progresses.
• release plan is flexible and expected to emerge based on
perceived business value.
27

28. Self-organising vs Planned
© 2013 CMMI Portugal Conference Series – All Rights reserved.
As the Practice Head – SEPG put it “there is nothing in CMMI
which states that teams cannot be self-organizing.”
So the balance between planned and self-organizing was
struck by emphasizing the common elements of goal-setting
and enabling some upfront design to support the distributed
context.
28

29. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Organisational Level
Standardisation of Processes
In SCRUM:
• it is commonplace to add engineering practices (e.g. XP)
• set of roles, practices and ceremonies defined is
rather small.
In the CMMI:
• large super-set of processes are defined;
• project teams must select practices from this
large set of process definitions.
29

30. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Need for Organisational Level
Standardisation of Processes – Case Study
When a project starts “we generally assign a coach to that
project” (Corporate Lead Architect)
To assess process compliance SEPG “will do monthly
reviews of the projects. And they also do audits to see if the
practices are being followed; are they following
continuous integration or are they making daily
builds” (Corporate Lead Architect)
30

31. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Project Metrics and Performance Prediction
• Metrics used for decision making and software
development process management to compare
between projects, spot trends, and anticipate
problems as well as for managing an individual
project.
• The data needs to be appropriate and part of the
normal process to minimize the effort of capturing
the data.
• CMMI recommends the use of a quantities model
to predict future performance. As long as the model
is specific to the project there is no contradiction.
31

32. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Project Metrics and Performance Prediction
– Case Study
• The company found “we faced problems in collecting data
required for level 4 KPA’s. The team felt that it was an
overhead.” (Delivery Head)
• “[The] most important aspect of complying with the metrics
needed for CMMI is to understand that traditional metrics
like, productivity and time slippage [are] not applicable …
instead we have been capturing metrics on defect,
resource utilization and on improvement.” (Practice Head –
SEPG)
32

33. Organisation level data base-lining
© 2013 CMMI Portugal Conference Series – All Rights reserved.
The problem in Agile is the consistency of data between
teams to enable comparisons and trends to be seen.
Traditional data sets are not relevant for agile projects
It takes time to develop a data set from which organisations
can predict future activity
33

34. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Organisation level data base-lining – Case
Study
The problem is that “some projects use story points and
some projects do not use story points. They use function
points” (Corporate Lead Architect)
The “initial problem with organizational metrics was that we did not
have sufficient data points … with more agile projects getting
executed that is getting resolved.” (Practice Head – SEPG)
SEPG defined an activity based estimation model. “So for example
you are saying this is a four story point for this feature... Okay so we
are trying to define some guidelines on the basis of what you will do
with your story point estimation. But that is against your agile
concept. It should be spontaneous [within any given team]”
(Program Head)
34

35. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Key Recommendations
35

36. Key Recommendations
© 2013 CMMI Portugal Conference Series – All Rights reserved.
• Establish key forms of record keeping techniques and
ensure they are auditable
36

37. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
37

38. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
• In larger settings, use an architecturally-driven agile
approach for goal setting in large distributed agile projects
balanced with local self-organisation for devolved design
38

39. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
• In larger settings, use an architecturally-driven agile
approach for goal setting in large distributed agile projects
balanced with local self-organisation for devolved design
• SEPG to use a team approach to define a process map for
agile projects that allows local tailoring, but also to add
elements such as risk management
39

40. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
• In larger settings, use an architecturally-driven agile
approach for goal setting in large distributed agile projects
balanced with local self-organisation for devolved design
• SEPG to use a team approach to define a process map for
agile projects that allows local tailoring, but also to add
elements such as risk management
• Develop a framework that enables the capture of data
relevant to their agile projects with consistent use of terms
40

41. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Summary
• Challenges in aligning CMMI and Agile but no major
contradictions
• Agile environments can mature – and this can be achieved
through using an SEPG team approach to agreeing
practices, data, etc
• Agile in an offshore, large-scale distributed context will
naturally deviate from ‘pure’ agile – making CMMI links
more natural
• We have identified some embryonic guidelines
• Just one major case – so need to extend the collection
41

42. © 2013 CMMI Portugal Conference Series – All Rights reserved.
Questions
42