The document discusses the experimentation-driven approach to organisational development. It argues that organizations should be viewed as complex adaptive systems and managed accordingly through an approach focused on experimentation. Experimentation is presented as a tool to manage uncertainty in complex domains where outcomes are unpredictable. The key aspects of experimentation discussed include developing hypotheses to test, designing prototypes and experiments, and reflecting on results to generate new knowledge and inform next steps.

1. sami@samipaju.com
THE EXPERIMENTATION-DRIVEN APPROACH TO
ORGANISATIONAL DEVELOPMENT
Spark the Change, June 13th 2017
#sparkto @sparkconfto
linkedin.com/in/samipaju@samipaju
SAMIPAJU.COM
samipaju
C O P Y R I G H T © S A M I PA J U 2 0 1 7

2. • Predictable cause and effect relationships
• Variables and their behaviour can be known
20th century and mechanistic thinking
• Systems can be broken into their constituent
parts, and those parts studied in isolation

3. “The society is like a clock whose secrets will be
revealed to us.
-Auguste Comte (1798-1857)
L O U T H , J O N AT H O N ( 2 0 1 1 ) . F R O M N E W T O N T O N E W T O N I A N I S M : R E D U C T I O N I S M A N D T H E D E V E L O P M E N T O F T H E S O C I A L S C I E N C E S . E M E R G E N C E : C O M P L E X I T Y &
O R G A N I Z AT I O N , V O L . 1 3 , N O . 4 , 6 3 - 8 3 .

4. DAVE CARROLL: UNITED BREAKS GUITARS

5. UNITED BREAKS GUITARS
IN 1980?
Local newspaper reports: “Airline breaks musicians
instrument – refuses to compensate!”

6. UNITED BREAKS GUITARS
IN 2010?
United Breaks Guitars -video is uploaded to YouTube.
Over a million views within the first three days.
United Airlines’ stock loses 10 % of its value (~180
million USD).

7. • Amount of interconnected objects
• Speed of information exchange
21st century and complexity thinking
• Unpredictability of chains of events
• Non-linear causal relationships

8. “Adaptability, not efficiency, must become our
central competency.
-Stanley McChrystal: Team of Teams

9. TO SURVIVE IN A COMPLEX WORLD REQUIRES EXPLOITATION AND EXPLORATION…
O ’ R E I L LY, C . A . , & T U S H M A N , M . L . ( 2 0 1 3 ) . O R G A N I Z AT I O N A L A M B I D E X T E R I T Y: PA S T, P R E S E N T A N D F U T U R E . A C A D E M Y O F M A N A G E M E N T P E R S P E C T I V E S , V O L . 2 7 , N O . 4 .
E X P L O I TAT I O N
O P E R AT I O N A L
E X C E L L E N C E I N
E S TA B L I S H E D
B U S I N E S S A R E A S
I N C R E M E N TA L
I N N O VAT I O N A N D
C O N T I N U O U S
I M P R O V E M E N T
E X P L O R AT I O N
C R E AT I O N O F N E W
V E N T U R E S A N D N E W
M A R K E T S
R A D I C A L A N D
D I S C O N T I N U O U S
I N N O VAT I O N

10. …BUT EXPLORATION DEMANDS AN ENTIRELY DIFFERENT MANAGEMENT APPROACH
E X P L O I TAT I O N
C E RTA I N A N D
P R E D I C TA B L E
C O N T E X T
E M P H A S I S O N
A N A LY S I S A N D
C O N T R O L
F O R M A L A N D
E X P L I C I T S T R U C T U R E S
E X P L O R AT I O N
U N K N O W N A N D
U N P R E D I C TA B L E
C O N T E X T
E M P H A S I S O N
C R E AT I V I T Y A N D
A G I L I T Y
N E T W O R K E D A N D
O P E N S T R U C T U R E S
O ’ R E I L LY, C . A . , & T U S H M A N , M . L . ( 2 0 1 3 ) . O R G A N I Z AT I O N A L A M B I D E X T E R I T Y: PA S T, P R E S E N T A N D F U T U R E . A C A D E M Y O F M A N A G E M E N T P E R S P E C T I V E S , V O L . 2 7 , N O . 4 .

11. “We are using mechanistic thinking and methods
in a world that is increasingly complex. To steer
organisations that do not behave mechanistically,
but instead follow the logic of complexity.
PA J U , S . ( 2 0 1 7 ) . K O M P L E K S I N E N M A A I L M A VA AT I I K E T T E R Ä Ä O R G A N I S O I T U M I S TA . I N M A RT E L A , F. , & J A R E N K O , K . ( E D S . ) I T S E O H J A U T U V U U S : M I T E N O R G A N I S O I T U A
T U L E VA I S U U D E S S A ? , H E L S I N K I : A L M A TA L E N T. C H A P T E R 2 .

12. ORGANISATIONS AS COMPLEX ADAPTIVE SYSTEMS
A N D E R S O N , P H I L I P ( 1 9 9 9 ) . C O M P L E X I T Y T H E O RY A N D O R G A N I Z AT I O N S C I E N C E . O R G A N I Z AT I O N S C I E N C E , V O L . 1 0 , N O . 3 , 2 1 6 - 2 3 2 .
1. Agents and schemata

13. ORGANISATIONS AS COMPLEX ADAPTIVE SYSTEMS
A N D E R S O N , P H I L I P ( 1 9 9 9 ) . C O M P L E X I T Y T H E O RY A N D O R G A N I Z AT I O N S C I E N C E . O R G A N I Z AT I O N S C I E N C E , V O L . 1 0 , N O . 3 , 2 1 6 - 2 3 2 .
1. Agents and schemata
2. Information flows between
agents

14. ORGANISATIONS AS COMPLEX ADAPTIVE SYSTEMS
A N D E R S O N , P H I L I P ( 1 9 9 9 ) . C O M P L E X I T Y T H E O RY A N D O R G A N I Z AT I O N S C I E N C E . O R G A N I Z AT I O N S C I E N C E , V O L . 1 0 , N O . 3 , 2 1 6 - 2 3 2 .
1. Agents and schemata
2. Information flows between
agents
3. System-level emergent
behaviour

15. COMPLEX ADAPTIVE… JAZZ?
E I S E N H A R D T, K . M . , & B R O W N , S . L . ( 1 9 9 8 ) . C O M P E T I N G O N T H E E D G E : S T R AT E G Y A S S T R U C T U R E D C H A O S . L O N G R A N G E P L A N N I N G , V O L . 3 1 , N O . 5 , 7 8 6 - 7 8 9 .

16. ORGANISATIONS AS COMPLEX ADAPTIVE SYSTEMS
A N D E R S O N , P H I L I P ( 1 9 9 9 ) . C O M P L E X I T Y T H E O RY A N D O R G A N I Z AT I O N S C I E N C E . O R G A N I Z AT I O N S C I E N C E , V O L . 1 0 , N O . 3 , 2 1 6 - 2 3 2 .
1. Agents and schemata
2. Information flows between
agents
3. System-level emergent
behaviour
4. Information flows between
systems, subsystems and
supersystems

17. ORGANISATIONS AS COMPLEX ADAPTIVE SYSTEMS
A N D E R S O N , P H I L I P ( 1 9 9 9 ) . C O M P L E X I T Y T H E O RY A N D O R G A N I Z AT I O N S C I E N C E . O R G A N I Z AT I O N S C I E N C E , V O L . 1 0 , N O . 3 , 2 1 6 - 2 3 2 .
1. Agents and schemata
2. Information flows between
agents
3. System-level emergent
behaviour
4. Information flows between
systems, subsystems and
supersystems
5. Constant change across
systems

18. EXPERIMENTATION IS A TOOL FOR COMPLEX DOMAINS
COMPLEX COMPLICATED
OBVIOUSCHAOTIC
The relationship between cause
and effect can only be
perceived in retrospect:
experimentation.
The relationship between cause
and effect requires investigation
or application of expert
knowledge: analysis & planning.
There is no relationship between
cause and effect at systems level:
rapid response.
The relationship between cause
and effect is obvious to all: best
practice.
S N O W D E N , D . J . , & B O O N E , M . E . ( 2 0 0 7 ) . A L E A D E R ’ S F R A M E W O R K F O R D E C I S I O N M A K I N G . H A R VA R D B U S I N E S S R E V I E W , N O V E M B E R 2 0 0 7 .

19. “Learning by experimentation is fundamental to
solving problems for which outcomes are
uncertain and where critical sources of
information are nonexistent or unavailable.
L E E , F. , E D M O N D S O N , A . C . , T H O M K E , S . , & W O R L I N E , M . ( 2 0 0 4 ) . T H E M I X E D E F F E C T S O F I N C O N S I S T E N C Y O N E X P E R I M E N TAT I O N I N O R G A N I Z AT I O N S . O R G A N I Z AT I O N
S C I E N C E , V O L . 1 5 , N O . 3 , 3 1 0 - 3 2 6 .

20. EXPERIMENTATION IS AN APPROACH FOR MANAGING UNCERTAINTY

21. STAND UP, HUG THE
PERSON NEXT TO YOU,
AND SMILE AS BIG AS
YOU CAN!
Raise your hand if you feel more
positive, energetic or happy!

22. WHAT MAKES AN EXPERIMENT?
A problem to solve or a goal to reach: perhaps
you are starting to feel a bit bored with my
monologue…
An experimentation idea: a quick exercise
where you have to move a little and interact with
other people.
Hypothesis (a fancy word for a guess): eye
contact, touch and smile will have a positive
effect on mood.
A way to collect feedback: how many hands
were up?
Reflection: what did we learn from the
experiment, and how does it affect what will be
done next?

23. THE GOAL OF AN
EXPERIMENT IS TO
CREATE AN EXPERIENCE
AND GENERATE NEW
KNOWLEDGE.

24. EXPERIMENTATION IS…
Behaviour
• Bias to action
I M A G E : VA L I A N T I Z E @ F L I C K R

25. EXPERIMENTATION IS…
Behaviour
• Bias to action
Mindset
• Skepticism towards second-hand
knowledge
• Trust on empirical evidence
• Willingness to dance with uncertainty
I M A G E : VA L I A N T I Z E @ F L I C K R

26. EXPERIMENTATION IS…
Behaviour
• Bias to action
Mindset
• Skepticism towards second-hand
knowledge
• Trust on empirical evidence
• Willingness to dance with uncertainty
A set of tools and techniques
• Prototyping
• Experiment design
• Sense-making
I M A G E : VA L I A N T I Z E @ F L I C K R

27. DESIGNING EXPERIMENTS
A step by step guide

28. A development
idea can be seen
as a set of claims
and assumptions.
IDENTIFYING UNCERTAINTIES: WHAT DO WE NEED TO LEARN FIRST?
S Y K E S , H . B . , & D U N H A M , D . ( 1 9 9 5 ) . C R I T I C A L A S S U M P T I O N P L A N N I N G : A P R A C T I C A L T O O L F O R
M A N A G I N G B U S I N E S S D E V E L O P M E N T R I S K . J O U R N A L O F B U S I N E S S V E N T U R I N G , 1 0 , 4 1 3 - 4 2 4 .

30. How do we create an
experience of our
development idea
and learn from the
experiment?
THE EXPERIMENT SETUP

31. The further an idea has
been developed, the
more refined the
prototypes tend to
become.
PROTOTYPE: A MEDIUM FOR LEARNING
H O U D E , S . , & H I L L , C . ( 1 9 9 7 ) . W H AT D O P R O T O T Y P E S P R O T O T Y P E ? I N H E L A N D E R , M . , L A N D A U E R , T. , & P R A B H U , P. ( E D S . ) H A N D B O O K O F H U M A N - C O M P U T E R I N T E R A C T I O N
( 2 N D E D . ) , A M S T E R D A M : E L S E V I E R S C I E N C E B . V.

32. ARE YOU TRYING TO
UNDERSTAND A
PROBLEM, OR ARE YOU
TESTING A SOLUTION?

33. EMPATHY PROTOTYPING: EXPERIENCING ANOTHER PERSON’S LIFE

34. SIMULATING SERVICES AND EXPERIENCES
M I N D R E S E A R C H G R O U P, A A LT O U N I V E R S I T Y, F I N L A N D

35. SIMULATING ENVIRONMENTS AND EXPERIENCES

36. CONVEYING YOUR IDEA WITH PAPER PROTOTYPES

37. EXPERIMENTING IN SAFE-TO-FAIL ENVIRONMENTS
I M A G E : M C G U I R K @ F L I C K R

38. CREATING SAFE-TO-FAIL EXPERIMENTS
M I N D R E S E A R C H G R O U P, A A LT O U N I V E R S I T Y, F I N L A N D

39. USING PROPS FOR BEHAVIOURAL EXPERIMENTS

40. A GREAT EXPERIMENT
CREATES AS MUCH
LEARNING AS POSSIBLE
WITH THE LEAST
AMOUNT OF EFFORT
Experiment design is as much a creative process
as coming up with good development ideas.

41. MINIMUM VIABLE EXPERIMENTS
M I N D R E S E A R C H G R O U P, A A LT O U N I V E R S I T Y, F I N L A N D

42. 1. What is the
hypothesis?
2. How does the
experiment benefit
the organisation?
3. How will you collect
feedback?
ORGANISED ACTION OVER LONG-TERM PLANNING
[target market / target users]
[will behave like this / will use the solution]
[because...]
Hypothesis Template
This Hypothesis Template is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.

43. 1. Questionnaires,
surveys and
interviews
2. Observation
3. Behavioural
evidence, metrics,
proof of commitment
COLLECTING FEEDBACK
PA L A U T E K A AV I O : W W W. K E H I TA K O K E I L L E N . F I / T Y O K A L U T /
Feedback Form
This Feedback Form is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.

44. Which assumptions
changed due to the
experiment and how
did they change?
What do you need to
learn next?
REFLECTION: WHAT DID THE EXPERIMENT TEACH US?

45. EXPERIMENTATION-DRIVEN APPROACH TO ORGANISATIONAL DEVELOPMENT
Development
goal
What are you trying to achieve?
Preventing
factors
Supporting
factors
Where will you intervene?
Development
ideas
How will you intervene?
Experimentation
ideas
Reflection
How will you experiment? What did you learn?
Whatwillyoudonext?
Experim
ents
Creative,divergent
thinking
A
nalytical,convergent
thinking
Development context Experiment design Knowledge creation

46. EXPERIMENTATION PITFALLS
Some things that typically go wrong

47. LACK OF COMMITMENT TO THE AGREED EXPERIMENTS

48. NO FEEDBACK COLLECTION, NO REFLECTION

49. Hypothesis
Notes
Notes
+ Where?
+ When?
+ Who will bring what
to the meeting?
+ Who?
+ How many?
+ How do we recruit them?
+ Responsible person
+ Where? Place
+ When?
Starting & ending dates
+ What? Tasks
+ Who? Responsible person
Item 1 + responsible person
Item 2 + responsible person
Item 3 + responsible person
+ How? Method
+ Where? Place
+ When?
+ Who? Responsible person
Execution:
We want to learn:
This is important for our
developmentidea,because:
This is important right now,
because:
Reflection meeting:
We will have learnt enough
when:
Preparingtheexperimentsetup:
Participants:
Required materials, props,
prototypes, etc.:
Collecting feedback:
purpose of the experiment
Task 1 + responsible person
Task 2 + responsible person
Task 3 + responsible person
NO PLANNING, NO COORDINATION

50. NO STRUCTURES, PROCESSES OR TOOLS THAT SUPPORT EXPERIMENTATION

51. CONTRADICTORY STRUCTURES, PROCESSES AND REWARDS

52. LACK OF LEADERSHIP SUPPORT AND ENCOURAGEMENT

53. LACK OF EXPERIMENTATION KNOW-HOW

54. BAD EXPERIMENTS GET MISTAKEN FOR BAD IDEAS

55. THE EMOTIONAL JOURNEY OF EXPERIMENTATION TAKES PEOPLE BY SURPRISE

56. “Make your decisions based on data, not
opinions. If you notice you’re arguing over
opinions, design an experiment. Get data, then
talk.
-Sami Honkonen, Tomorrow Labs

57. YOUR EXPERIMENTATION TOOLKIT: BIT.LY/SPARK2017TOOLS
SAMIPAJU.COMCopyright © Sami Paju 2017
Idea Canvas
This Idea Canvas is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.
[target market / target users]
[will behave like this / will use the solution]
[because...]
Hypothesis Template
This Hypothesis Template is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.
Uncertainty Matrix
This Uncertainty Matrix is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.
Hypothesis
Notes
Notes
Notes
+ Where?
+ When?
+ Who will bring what
to the meeting?
+ Who?
+ How many?
+ How do we recruit them?
+ Responsible person
+ Where? Place
+ When?
Starting & ending dates
+ What? Tasks
+ Who? Responsible person
Item 1 + responsible person
Item 2 + responsible person
Item 3 + responsible person
+ How? Method
+ Where? Place
+ When?
+ Who? Responsible person
Execution:
We want to learn:
This is important for our
developmentidea,because:
This is important right now,
because:
Reflection meeting:
We will have learnt enough
when:
Preparingtheexperimentsetup:
Participants:
Required materials, props,
prototypes, etc.:
Collecting feedback:
purpose of the experiment
Task 1 + responsible person
Task 2 + responsible person
Task 3 + responsible person
Experimentation Plan
This Experimentation Plan is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.
Execution:
We want to learn:
This is important for our
developmentidea,because:
This is important right now,
because:
Reflection meeting:
We will have learnt enough
when:
Preparingtheexperimentsetup:
Participants:
Required materials, props,
prototypes, etc.:
Collecting feedback:
Experimentation Plan
This Experimentation Plan is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.
purpose of the experiment
Feedback Form
This Feedback Form is originally published in www.kehitakokeillen.fi
and licensed under a Creative Commons Attribution-ShareAlike 4.0 International license:
http://creativecommons.org/licenses/by-sa/4.0/.
SPARK THE CHANGE, June 13th 2017
#sparkto @sparkconfto
Mechanistic view of organisations is helplessly outdated
in modern and fast-changing business environments.
Organisations need both exploitation and exploration to
survive. The faster the change, the more emphasis
should be put on exploration.
A great experiment creates as much learning as possible
with the least amount of effort.
The goal of an experiment is to create an experience
and generate new knowledge.
All development ideas consist of claims and
assumptions. Create experiments around those that are
most important and uncertain.
sami@samipaju.com linkedin.com/in/samipaju@samipaju samipaju