In this session, I talk about how empirical data (which I’m calling agile numbers) can serve as signposts on our journey to reach the goal of delivering customer value faster with maximum reliability and minimum issues. I’ll show how agile numbers can help us determine if a practice is worth the investment and whether it will improve our team’s performance. I will also share examples of agile numbers that can assist you in your agile journey. Those numbers are pulled from different sources like scientific studies, research from Google, and the 2017 State of DevOps Report. The talk explores what empirical data tells us about what works and what doesn’t in software delivery: * What is the best team structure that leads to high IT performance? * What is the best architecture that leads to high IT performance? * Do skills, experience and personalities on a team what distinguishes great teams? * What leadership attributes are tied to high IT performance? Is estimation scientifically valid? * Do code review works? How much time should we spend on them? * How long is too long for a branch? * And much more

1. AGILE
NUMBERS_
The Signposts on Your Agile Journey
By Akrem Saed

2. The name
that I
really
wanted_

3. Current
State of
Agile
Practices_

4. Agile didn’t
start as a
“landscape”
agilemanifesto.org

5. Why
Data?_
agilemanifesto.org

6. Good
Questions_
Which one is more popular?
Which one is endorsed by my agile coach?
Which one did Netflix use?
Which one can scale to the size of my organization?
Which one is easy to pick up?
?

7. Which ones
have data
behind them?
?
Better
Question_

8. Current
State_
of Discourse

9. “Google has been using it for years”
“I’ve used it in a past project and worked wonders”
“Our competitors are using it “
“I’ve read this blog post”
“It’s trending on Twitter/publications/conferences”
“It made sense to me!”
Subjective
Measures_

10. Controlled Scientific Studies
Statistical Analysis of IT Behavior
Comparative Case Studies
Large-Scale Experiments
Small-Scale Experiments
Objective Measures_
(a.k.a Agile Numbers)

11. Scientific Studies
& Research_

12. What We Know About
Software Engineering_
“Bits of Evidence:
What We Actually Know About Software
Development, and Why We Believe It’s
True”
- Greg Wilson, January 2010
https://vimeo.com/9270320

13. Standard
Of Proof_
“.. the debate is hampered because not enough
people know how to develop DSLs effectively”
- Martin Fowler, 2009
“One of the smartest guys in our industry made
two substantive claims in an academic journal
without a single citation ….. I think the debate
is hampered by low standards of proof”
- Greg Wilson, 2010

14. Raise the Standard:
Studies_
Anchoring and Adjustment in Software Estimation
(Aranda and Easterbook 2005)
Finding:
the anchor mattered more than experience,
how formal the estimation method was,
or anything else.

15. (van Genuchten 1991 and many others)
The two biggest causes of project failure are poor estimation and unstable requirements.
(Woodfield, 1979)
For every 25% increase in problem complexity, there is a 100% increase in solution complexity.
(Thomas et al, 1997)
If more than 20-25% of a component has to be revised, it’s better to rewrite it from scratch.
Raise the Standard:
Studies_

16. (Fagan 1975)
Rigorous inspections can remove 60-90% of errors before the first test is run.
(Nagappan et al (2007) & Bird et al (2009)
Physical distance does not affect post-release fault rates. Distance in the organization chart does.
Raise the Standard:
Studies_
(Kemerer and Paulk 2009)
200 LOC/hour or less is an effective rate for code reviews, identifying more than half of the defects

17. Want more
studies?_
Making Software: What Really
Works, and Why We Believe it
By Andy Oram & Greg Wilson
https://goo.gl/aPUzGo

18. Erdogmus and Williams 2003
Pairs passed 15% more of the automated post-development test cases.
Salleh et al 2009
Pair programming creates an environment conducive to more advanced learning and social
interaction, leading to students being less frustrated, more confident and more interested in IT
More Studies:
Pair Programming_
Erdogmus and Williams 2003
Average difference in programmer-hours between pairs and solo devs was not statistically significant.

19. Chong and Hurlbutt 2007
Engineers seemed more engaged when in possession of keyboard or perceived control was imminent
Vanhanen and Kapori 2007, Vanhanen
and Lassenius 2007
Pair programming contributed to team morale, increased discipline in use of other prescribed
practices (e.g. TDD, coding standards, and frequent integration)
More Studies:
Pair Programming_
Arisholm et al 2007
Large experiment demonstrated a quality improvement with pairs on complex tasks but not on
simpler tasks

20. Folk Medicine
in IT_
Study What Already
Works in the Wild

21. Project
Aristotle_
Quest to Build
The Perfect Team

22. Aristotle:Test the
commonly-held myths_
‘‘... there was nothing showing that a mix of
specific personality types or skills or
backgrounds made any difference. The ‘who’
part of the equation didn’t seem to
matter.’’
- Abeer Dubey, leader of Project Aristotle
New York Times > https://goo.gl/z1jjac

23. Aristotle:
The Safety Hint_
What distinguished the “good” teams
from the dysfunctional groups was
how teammates treated one another.
Evidence for a Collective Intelligence Factor in
the Performance of Human Groups
Woolley et al. (2010)

24. “As long as everyone got a chance to talk, the
team did well, But if only one person or a
small group spoke all the time, the collective
intelligence declined.’’
- Anita Williams Woolley
Equal Talk Time1
Aristotle:
The Safety Hint_

25. Teammates were skilled at figuring out how
others felt based on their tone of voice,
expressions and other nonverbal cues.
2
Aristotle:
The Safety Hint_
High average social sensitivity

26. Aristotle: 1+1
Psychological Safety_
“[Psychological safety is] a sense of
confidence that the team will not embarrass,
reject or punish someone for speaking up,’’
- Amy Edmondson (1999)
TEDx Talk > https://goo.gl/eXbxXn

27. Aristotle:
The Finding_
“There were other behaviors that seemed
important as well — like making sure teams
had clear goals and creating a culture of
dependability. But Google’s data indicated
that psychological safety, more than
anything else, was critical to making a
team work. ”

28. More About
Project Aristotle_
New York Times > https://goo.gl/z1jjac Google re:Work > https://goo.gl/19k4u7

29. Surveys &
Statistical
Models_

30. 27,000 Survey responses
3,200 People
6 Years
DevOps Report:
The Data_

31. DevOps Report:
Diverse Data Set_
Demographics
DevOps teams increased
● from 16% in 2014
● to 19 % in 2015
● to 22% in 2016
● to 27% in 2017.
DevOps Report 2017 > https://goo.gl/yqTHvz

33. DevOps Report:
Diverse Data Set_
Respondent by Regions

34. DevOps Report:
IT Performance_

35. Stability gap
Throughput gap
High
Performing Teams
Vs
Low
Performing Teams
DevOps Report:
IT Performance Gap

36. DevOps Report:
IT Performance Gap

37. The High
Performing Teams’
Advantages
High responsiveness to
market changes
Better customer experience
Faster time to market
DevOps Report:
IT Performance Gap

38. DevOps Report:
Technical Practices_

39. 33% more of configuration management
27% more of testing
30% more of deployments
27% more of change approval processes
High
Performing Teams
Vs
Low
Performing Teams
DevOps Report:
Automation

40. Medium- were doing more manual work
than low-performing teams in
DevOps Report:
Automation

41. “Continuous Delivery
significantly
contributes to both
lower deployment
pain and higher IT
performance”
DevOps Report:
Continuous Delivery

42. In 2015, high-performing teams were more likely to have
loosely coupled architecture.
● Loosely Coupled Teams contribute to better IT
performance.
● Loosely Coupled Architecture derives IT performance.
■ Bounded Contexts (Domain Driven Design)
■ APIs
■ Test doubles for testing
In 2017,
DevOps Report:
Architecture

43. Biggest contributor to Continuous Delivery (bigger
than test and deployment automation) is if a team can
do:
● Make design changes without permission or dependency
● Complete its work without involving other teams
● Deploy and release on demand
● Do most of testing on demand without integrated test
environment
● Deploy during business hours with negligible downtime
DevOps Report:
Continuous Delivery &
Teams

44. 2017
High-performing teams spent
21% less time on unplanned work and
rework
44% more time on new features or code
2016
High-performing teams spent
22% less time on unplanned work and
rework
29% more time on new features or code
DevOps Report:
Quality

45. In 2017, report analyzed how long
branches live and time to integrate
branches into trunk
High-performing teams spend only hours on
both measures
Low-performing teams spend days
In 2016, higher software delivery
performance contributed by:
Merging code into trunk on daily basis
Having short-lived branches and forks (less than 1
day)
Having fewer than three active branches
DevOps Report:
Trunk-Based Dvlpmnt._

46. DevOps Report:
Lean Product
Management_

47. In 2017
the report found IT performance predicts lean product
management practices
In 2016
lean product management practices predicted:
● Higher IT performance
● Lower deployment pain
DevOps Report:
Lean Product
Management_
2016 + 2017
A virtuous cycle between LPM and software
delivery pipeline

48. Impact of Leadership
DevOps Report:
Impact of Leadership_

49. Statistically significant differences
between high-performing and
low-performing teams when it
comes to 5 leadership characteristics
DevOps Report:
Transformational Leader

50. Teams with least transformative
leaders are 50% less likely to exhibit
high performance
DevOps Report:
Transformational Leader

51. DevOps Report:
Transformational Leader

52. Objective Team
Transformation
How should your team evolve?

53. Dig Deeper1
Objective Team Transformation

54. Pick a Practice or
Insight from Here2
Objective Team Transformation

55. Favor Objective
Measures3
Objective Team Transformation

56. Measure Impact
on Your Team4
Objective Team Transformation

57. Act on
Measured Impact5
Objective Team Transformation

58. @AkremSaed
Who was this
Akrem dude ?