Jim McHale presented on the relationship between Agile and CMMI. While they have different origins, focusing on manufacturing versus knowledge work, they also have similarities in emphasizing iterative processes. Both aim to build processes tailored to people rather than having people follow rigid processes. McHale discussed how early testing through inspection can find most defects, saving more time than other testing methods. He encouraged QA professionals to provide data-driven process improvement through measurement and disciplined changes based on feedback.

1. Agile Meets CMMI… Again
Notes from Tes Trek conference -
Keynote presentation by
James McHale
November 8th 2012
“Notes from Tes Trek conference” by Maira Bay de Souza is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License

2. Before we begin ...
Items in this font are the notes I took of what Jim
said
Items in this font are my own comments

3. Introduction
Process-oriented people have always tried to be more
Agile, while Agile defenders are constantly trying to be
more process-oriented!
Interesting comment. I had never thought of it that way, but I think
he is right.
Good process professionals will build a process that works for the
people, instead of having people work for the process. That means no
useless paperwork or bureaucracy. Only do what is essential. And
isn't that what Agile is about?

4. About SEI and CMMI
SEI works on the state of the practice, not the state of the art
CMMI will soon move out of SEI and into a new organization called the
CMMI Institute. There will be little or no change on how you work
with them and on the contracts themselves.
State of the art = latest research on process improvement
State of the practice = what companies are actually doing
I have studied about process and then worked in organizations of all sizes. I
agree that there is a difference between the best thing that companies
should be doing (according to research) and what they are actually doing.
From what Jim said, it seems like this new organization will be more “for
profit” than SEI. There isn't much info on the web yet about this
transition. The best place I could find was here:
http://sei-pab-chair.blogspot.com/

5. CMMI and Agile - similarities
CMMI and Agile have roots in common: Agile was
developed after CMMI because CMMI was too
heavyweight for some organizations.

6. CMMI and Agile - differences
CMMI was developed based on ideas of authors that came
from a manufacturing background (see recommended
readings 1, 2 and 3)
The people who developed Agile were more in line with the
thoughts of Peter Drucker. He was more focused on the
knowledge industry.
Very interesting remark. I have heard a few people in the software
community say that we shouldn't take a manufacturing approach to
software development. We are after all building a product based on
information. We are not manufacturing tangible goods.
I hope to learn more about the connection between Peter Drucker and
Agile, and then share it with you!

7. Other SDLCs
The original author of the Waterfall model (see
recommended reading 5) intended it to be iterative. But
the DOD ended up implementing it as a one-time
iteration.
Agile is very similar to TSP, making it closer to CMMI than
we would think.
Interesting comment about the waterfall model. It relates to
what he said before about state of the art and state of the
practice: what the author intended was not what ended up
being done.

8. Organizations
Most organizations adapt themselves to look like
their products and not the other way around
(unfortunately)
I agree with him. It is unfortunate but true. I've seen many
organizations that work around their systems instead of
changing or modernizing their systems to make them more
efficient and customer-focused.

9. BDUF
BDUF = Big Design Up-Front
This type of software life-cycle was not
created because of CMMI. It was
created because of the DOD's
competition rules. Design had to be
ready before the competition started.
Ah, that explains a lot!
It's amazing how much the DOD has shaped
the world of software development.

10. New trend
5 years ago the DOD noticed that CMMI L3 to L5
companies had no difference in the performance
between them.
Interesting. If I were to make a guess I would say that L5 companies
aren't following their processes as strictly as they should, hence
performing as L3. But that's just a guess. I wonder if they found a
reason for this, and what the reason was.

11. Contradiction
Some people in the Agile world are very strict on their
processes, which is funny, because that was the whole
point of Agile in the first place ( = to not follow any
process!)
A company called Systematic is following Agile, but has
lots of data (just like a CMMI L4 company)
Well, what can I say... humans are contradictory!
By the way, I don't have anything against the Agile people. I
believe that whatever works for an organization is what they
should use - as long as it will still keep on working in the
short to medium term future.

12. Kelvin's quote
“When you can measure what you are speaking about, and
express it in numbers, you know something about it,
when you cannot express it in numbers, your knowledge
is of a meager and unsatisfactory kind; it may be the
beginning of knowledge, but you have scarely, in your
thoughts advanced to the stage of science.”
Measurement is the foundation for improvement
Measurement is the act of reducing uncertainty
Look at recommended reading 7

13. Metrics and QA
QA has most of the data about an SDLC
I go to organizations and the first place I look for data is
the QA department
Sometimes it's the first time anyone has ever asked to see
the data they have been collecting for so long
I'm surprised and disappointed by that information. The data
collected by QA should be used in improving the process. If
the organization is not using the data, then they are wasting
the QA team's time and consequently their own money.
As I love to say it: the process should work for the people, and
not the people work for the process. The case Jim describes
is a classical case of people working for the process. That
means huge waste of resources, time and money.

14. QA and defects
In a project where testing is only done at the end (i.e.,
using waterfall SDLC), the # of defects found/hour varies
a lot. Example: QA spent 5 hours of test execution and
didn't find any defects. Then, in the next 30 min they
found 14 defects.
In a project with early testing (using V-Model SDLC), the #
of defects found/hour is more constant. Example: QA
found 1-2 defects every 30 min.
Interesting observation. That tells me that QA will be
perceived as more efficient with early testing.

15. Scenario Coverage
Did you know? When Microsoft tests Windows, they only
cover 0.1% to 2% of the possible scenarios!
That is because the PC has too many possible
configurations (think of all the different brands of
motherboard, video card, mouse, etc that exist).
Apple computers, on the other hand, have much less
variability. That's why they are able to test a much
bigger % of the configuration possibilities.
Micheal Fagan (see recommended reading 9) started doing
inspection in the 60s!
A-ha! That explains a lot!

16. Code inspection
The latest trend in testing is having the testers fix the
code themselves, during code inspection.
Less than 50% of organizations do inspection. Of those,
less than 50% do it effectively.
Even CMMI doesn't value inspection that much. It only
allows you to inspect 10% of the code.
Inspection saves more time than any other testing method.
It's better to do only 100% inspection than to do other
methods of testing.
Lots of food for thought on inspection here.
I think what he is saying makes sense. After all, the earlier the
testing, the better. And if we find all the defects in inspection,
then it's true: we don't need other types of testing.
That is quite a revolutionary idea, but definitely something to think
about (and maybe experiment in your organization!).

17. Defect-free cycle
How long should a full defect-free test
cycle last (including test design)?
The answer to that question will be how
much the organization is saving by
doing inspection
What a smart way to measure the value of
inspection!

18. Call to Arms!
For all QA professionals:
- Bring discipline to development organizations (not
punishment!)
- Measurement is essential
- Encourage disciplined process changes (whatever your
process is) based on feedback given by the
data/measurement
- If you have many testing cycles, use one of them to
inspect the code
Great presentation! Great tips!
I learned a lot from Jim. I hope those of you who were
unable to attend the conference have learned a lot from
my notes too!

19. Recommended Readings
[1] Quality is Free - Philip B. Crosby
[2] works of Watts Humphrey
[3] The Knowledge-Creating Company - Ikujiro Nonaka and Hirotaka Takeuchi
[4] Software Engineering Best Practices - Capers Jones
[5] works of Winston Royce
[6] Jeff Sutherland's articles on MSDN:
http://msdn.microsoft.com/en-us/library/dd997578(v=vs.100).aspx
http://msdn.microsoft.com/en-us/library/hh350860(v=vs.100).aspx
[7] How to Measure Anything - Douglas W. Hubbard
[8] The Checklist Manifesto - Atul Gawande
[9] works of Michael Fagan

20. Disclaimer
The notes presented here are what I understood from what Jim
communicated. They might not be 100% accurate, as I was taking
notes and listening to the presentation at the same time.
All the information I am quoting from Jim is his intellectual property.
I am reproducing it here under the fair use policy, for quoting
purposes only.