How can complexity science be applied to software development? This presentation shows you which scientific concepts can be mapped to agile software development. http://www.noop.nl http://www.jurgenappelo.com

1. What (Else) Can
Agile Learn from
Complexity?
What (Else) Can
Agile Learn from
Complexity?
Jurgen Appelo
jurgen@noop.nl
version 3
image by BotheredByBees
Jurgen Appelo
jurgen@noop.nl
version 3

2. Jurgen Appelo
writer, speaker,
entrepreneur...
www.jurgenappelo.com

3. QDD (Quote-Driven Discourse)
Misquotation is, in fact, the pride and privilege of the
learned. A widely-read man never quotes accurately,
for the rather obvious reason that he has read too
widely.
Hesketh Pearson (1934)
Common Misquotations
http://www.amazon.com/Common-Misquotations-Hesketh-Collected-Pearson/dp/B00128UERS/
“
”

4. Let’s start!Let’s start!

5. Agile Software Development
An ant colony, the brain, the immune system,
a Scrum team, and New York City, are
self-organizing systems.
Ken Schwaber
Agile Software Development with Scrum
http://www.amazon.com/Agile-Software-Development-Scrum/dp/0130676349/
“
”

6. Agile Software Development
At the heart of complex adaptive systems theory’s
relevance to software development is the concept
of emergence, and the factors leading to emergent
results.
Jim Highsmith
Adaptive Software Development
http://www.amazon.com/Adaptive-Software-Development-Collaborative-Approach/dp/0932633404/
“
”

7. Agile Software Development
Scrum is not a methodology, a defined process or
set of procedures. It's an open development
framework. The rules are constraints on behavior
that cause a complex adaptive system to self-
organize into an intelligent state.
Jeff Sutherland
Shock Therapy (presentation)
“
”

8. The Evolution of Complexity Science
image by Brian Castellani

10. Are software projects complex systems?
Can we apply complexity science to management?
image by practicalowl

11. Beware!! Abuse of Science
1. To use scientific terminology without bothering much
about what the words actually mean.
2. Importing scientific concepts without giving the slightest
empirical or conceptual justification.
3. Displaying superficial erudition by shamelessly throwing
around terms where they are irrelevant.
4. Exhibiting a veritable intoxication with words, with a
superb indifference to their meaning.
Sokal, A. and Bricmont, J. (1998)
Intellectual Impostures: Postmodern Philosophers’ Abuse of Science
http://www.amazon.com/Intellectual-Impostures-Alan-Sokal/dp/1861976313
“
”

12. Complexity vs. Management
Not unexpectedly, the complexity gurus are
most upset with how complexity science terms are
loosely, if not metaphorically, defined and tossed in
managerial discourse – one [guru] goes as far as to
suggest that the books offer many insights for
managers, but one should simply black out all
references to complexity science.
Maguire, S. and McKelvey, B. (1999)
“Complexity and Management: Moving from fad to firm foundations”
http://iscepublishing.com/ECO/ECO_papers/Issue1_2_2.pdf
“
”

13. Compromise: Loose Application
The word “culture” in business culture was (successfully)
borrowed from anthropology.
The word “strategy” in business strategy was borrowed from
the military. (In fact, the science of game theory borrowed
the word for evolutionary stable strategies!)
Loose application (borrowing and bending) of terminology
across disciplines is more than just the invention of
metaphores. The terms actually have meaning.
Van Uden, J. (2005)
“Using complexity science in organization studies: a case for loose application”
http://iscepublishing.com/ECO/ECO_papers/Issue_7_1_8_PH.pdf

14. Snowden, D. and Stanbridge, P. (2004)
“The Landscape of Management: Creating the context for understanding social complexity”
http://iscepublishing.com/ECO/ECO_other/Issue_6_1-2_19_FM.pdf

15. humans involved,
not mechanics
complex,
not ordered
Snowden, D. and Stanbridge, P. (2004)
“The Landscape of Management: Creating the context for understanding social complexity”
http://iscepublishing.com/ECO/ECO_other/Issue_6_1-2_19_FM.pdf

16. Example: Self-Organization
Self-organization is a process of attraction and
repulsion in which the internal organization of a
system, normally an open system, increases in
complexity without being guided or managed by an
outside source.
Self-Organization
http://en.wikipedia.org/wiki/Self-organization
Are self-organizing teams really self-organizing?
“
”

17. Example: Emergence
Emergence is the arising of novel and coherent
structures, patterns and properties during the process
of self-organization in complex systems.
Corning, P.A. (2002)
“The Emergence of Emergence: Now What?”
http://iscepublishing.com/ECO/ECO_papers/Issue4_3_3.pdf
Is emergent design really emergent?
“
”

19. Software Projects
Scope
Features
Qualities People
Tools
Resources
Parts
Artifacts
Product

20. Yes, we can apply complexity to management
Yes, software projects are complex systems
image by Peter Kaminsky

21. Darkness Principle
Each element in the system is ignorant of the
behavior of the system as a whole [...] If each element
‘knew’ what was happening to the system as a whole,
all of the complexity would have to be present in that
element.
Richardson, K.A. (2002)
“Systems theory and complexity: Part 1”
http://iscepublishing.com/ECO/ECO_other/Issue_6_3_10_FM.pdf
“
”

22. Traditional Monitoring and Control fails
(no project manager can know everything)
image by Okko Pyykkö

23. The Law of Requisite Variety
If a system is to be stable the number of states of its
control mechanism must be greater than or equal to
the number of states in the system being controlled.
Variety (Cybernetics)
http://en.wikipedia.org/wiki/Variety_%28cybernetics%29
“
”

24. Traditional Management fails
(one manager is less complex than a group)
image by flattop341

25. Boundaries and Conditions
Self-organization requires that the system is
surrounded by a containing boundary. This condition
defines the "self" that will be developed during the
self-organizing process.
Eoyang, G. and Conway D.J. (1999)
“Conditions That Support Self-Organization in a Complex Adaptive System”
http://amauta-international.com/iaf99/Thread1/conway.html
“
”

26. image by Robbie W T
Agile Management is a crucial part of Agile
(Defining boundaries and conditions)

27. Hierarchy Principle
Complex natural phenomena are organized in
hierarchies wherein each level is made up of several
integrated systems.
Skytner, L. (2001)
“General Systems Theory: Ideas and Applications” NJ: World Scientific
“
”

28. image by Mountain Goat Software
Example: Scrum of Scrums
managed self-organized
Hierarchies are not a problem, they are natural
Managed (imposed) hierarchies are the problem

29. Patches
Take a conflict-laden task in which many parts
interact, and divide it into a quilt of non-overlapping
patches. Try to optimize within each patch. As this
occurs […] a good solution in one patch will change
the problem to be solved in adjacent patches.
Kaufman, S. (1995)
At Home in the Universe
http://www.amazon.com/At-Home-Universe-Self-Organization-Complexity/dp/0195111303/
“
”

30. image adapted from Mountain Goat Software
Example: Patches of Scrums
An ecosystem of teams instead of a hierarchy

31. Group Size
There is a particular number of decision-makers that
stands out from the trend as being truly,
spectacularly bad, tending with alarmingly high
probability to lead to deadlock: eight.
Buchanan, M. (2009)
“Explaining the curse of work”
http://www.newscientist.com/article/mg20126901.300-explaining-the-curse-of-work.html
“
”

32. image by Jayel Aheram
Good team size = 7 +/- 2 (except 8)
(5 might be perfect)

33. Specialization
In more complex organisms – from eukaryotic
protists to social mammals – specialization among
the parts (or division of labor) produces many
advantages for various wholes.
Corning, P.A. (2002)
“The Emergence of Emergence: Now What?”
http://iscepublishing.com/ECO/ECO_papers/Issue4_3_3.pdf
“
”

34. Specialization is not a problem, it is natural
Managed (imposed) specialization is the problem
image by Ali

35. Power Laws
Scale-free networks are ubiquitous in nature due to
the adaptive properties of robustness and
fast communication associated with power-law
degree distributions.
Mitchell, M. (2009)
Complexity: A Guided Tour
http://www.amazon.com/Complexity-Guided-Tour-Melanie-Mitchell/dp/0195124413/
“
”

36. Power laws exhibit scale invariance
You can’t estimate size of next unknown issue
size of issues
frequency
of issues
high
chance
of small
issues
low
chance
of large
issues

37. Prediction of velocity includes an (impossible)
estimate of the size of unknown problems
image by cod gabriel

38. Dependence on Context
A project cannot be viewed independent of its
surrounding context […]. An understanding of the
context is in itself not sufficient to prescribe a method
[…]. Rather, the method to manage the project is
embedded in the context and one must allow the
emergence of such a method through interaction
between the actors and the environment.
Pundir, A.K., Ganapathy, L. And Sambandam, N. (2007)
“Towards a complexity framework for managing projects”
http://iscepublishing.com/ECO/ECO_papers/Issue_9_4_3_AC.pdf
“
”

39. ScrumButs are natural and necessary
Management methods must emerge
Standard methods must be customized

40. Fitness Landscapes
The environment is not out there, separate from us.
We can help to create the environment. […] The
Spanish have phrase […]: “My friend, there is no
road. You make the road as you walk.”
Dent, E.B. (1999)
“Complexity Science: A Worldview Shift”
http://iscepublishing.com/ECO/ECO_papers/Issue1_4_1.pdf
“
”

41. The environment responds to released software
Success can only be known after deployment
image by neona

42. ?
linear improvement
stuck on local optimum
or walk through the valley
(linear) retrospectives may stop working

43. image by Kevin N. Murphy
mixing practices from two sources
Crossover

44. non-linear improvement
finding higher peaks by
mixing prior high performers
Switch from linear to non-linear retrospectives
(Replace multiple practices at a time)

45. moderately connected system
(several optimums)
sparsely connected system
(one optimum)
intensively connected system
(no real optimum)

46. If you can’t replace a practice, tool, or person,
you shouldn’t even add them in the first place
image by wilhei55

47. disturbances enable the system
to find deeper valleys
(inverted landscape, lower = better)
Noise

48. image by Hamed Saber
The Butterfly Effect
Small changes, BIG consequences

49. Incompressibility
There is no accurate representation of the system
which is simpler than the system itself. In building
representations of open systems, we are forced to
leave things out, and since the effects of these
omissions are nonlinear, we cannot predict their
magnitude.
Cilliers, P. (2005)
“Knowing complex systems”
Managing Organizational Complexity: Philosophy, Theory And Application
http://www.amazon.com/Managing-Organizational-Complexity-Philosopy-Application/
“
”

50. Because of the butterfly effect, we cannot predict
the consequences of leaving something out
Consequence: a variety of imperfect methods
image by woodleywonderworks

51. Models
All models are wrong, but some are useful.
Box, G. and Draper, N. (1969)
Evolutionary Operation: A Statistical Method for Process Improvement
http://www.amazon.com/Evolutionary-Operation-Statistical-Process-Improvement/dp/B0028H5YDG/
“
”

52. The Software Cultivator
Managing Agile Organizations
(2010)
?
Jurgen Appelo
My Book

53. Disclaimer
I might have used scientific terminology without
really understanding what the words actually mean.
I might have imported scientific concepts without
giving sufficient justification.
I might have thrown around scientific terms where
they are irrelevant.
And I might have an intoxication with words...

54. image by -bast-
But I hope to have
inspired you...
Q & A

55. m30.me/happiness

56. @jurgenappelo
slideshare.net/jurgenappelo
noop.nl
linkedin.com/in/jurgenappelo
jurgen@noop.nl

57. http://creativecommons.org/licenses/by-nd/3.0/
This presentation was inspired by the works of many people, and
I cannot possibly list them all. Though I did my very best to attribute
all authors of texts and images, and to recognize any copyrights, if
you think that anything in this presentation should be changed,
added or removed, please contact me at jurgen@noop.nl.