Slides as presented at the 2019 Agile Midwest Conference.
https://agilemidwest.org
Abstract: Hypothesis-Driven Development is thinking about the development of new ideas, products and services – even organizational change – as a series of experiments to determine whether an expected outcome will be achieved, so we need to know how to design and run experiments properly.
This session helps participants understand the importance of using experiments to help teams and organizations learn and improve, while giving hands-on practice in designing experiments to yield measurable evidence for that learning.
4. In complex environments, you can’t follow recipes or
conduct detailed analysis to understand the situation.
Rather, you must experiment (probe).
-- Cynefin
Improve collaboratively, evolve experimentally.
-- Kanban Method
Pivoting …. a structured course correction designed to
test … hypothesis about the product, strategy, and engine
of growth.
-- Lean Startup
Each feature includes a benefit hypothesis.
-- SAFe
WHYEXPERIMENTS?
@mattphilip #AgileMW2019
6. IDEA
◆ This is a variation on Eleusis Express; a card game of inductive reasoning which itself is a
modification of Robert Abbott’s game Eleusis.
◆ One player has a secret rule for which cards can be played.
◆ For example: Each card has to be a different color from the card before it. Other players lay
down cards to discover the pattern. If a player lays down a card correctly, he or she can try to
guess the rule.
@mattphilip #AgileMW2019
7. OBJECT
◆ A game consists of one or more rounds/hands.
◆ A different player is chosen as dealer of each new round, and it is the dealer who knows the
secret rule. The dealer does not play a hand that round.
◆ All plays are made to a central layout that grows as the round progresses. A layout consists of
a horizontal mainline of correct cards that follow the secret rule. Below the mainline are
vertical sidelines of mistake cards that did not follow the rule. These sideline cards can be
overlapped to save space.
@mattphilip #AgileMW2019
8. DEALER
◆ The dealer is given a secret rule.
◆ He or she places the pile of cards face down, turns over the top card, and puts it on the table.
This will be the start of the mainline.
◆ Before play starts, the dealer may give a hint about the rule.
◆ The player to the left of the dealer goes first, then the play continues around to the left.
@mattphilip #AgileMW2019
9. PLAYERS
◆ A player turns the top card from the deck and lays it on the table. He or she asserts whether it
is correct or not, and the dealer confirms or denies.
◆ If correct, the card goes to the right of the last card on the mainline.
◆ If incorrect, the card goes below the last card (it either starts a sideline or it adds to a sideline).
@mattphilip #AgileMW2019
10. PLAYERS (NO PLAY)
◆ A player has the option of declaring that they have no correct or incorrect cards to play. They
shows their hand (to everyone) and the dealer says whether the player is right or not.
◆ If the player is wrong, the dealer chooses one correct or incorrect card from their hand which
could have been played, and puts it on the layout. For example if the player declares they
have no correct cards, a correct card would be chosen. The player keeps their hand and they
must draw one card from the stock.
◆ If the player is right, and their hand is down to one card, that card is put in the stock and the
round is over. If they have more than one card, the dealer counts their cards and puts them on
the bottom of the stock. They then deal the player a new hand, but with one less card.
@mattphilip #AgileMW2019
11. GUESSING
◆ Whenever a player makes a correct assertion, he or she can guess the rule.
Everyone must hear the guess.
◆ The dealer then says whether the player is right or wrong. If wrong, the game
continues. If right, the round ends.
@mattphilip #AgileMW2019
14. 1. Choose who is who. One player becomes the codemaker, the others are the
codebreakers
2. Choose 4 digits. The codemaker chooses a pattern of four digits; for
example 1 2 3 4.
3. Try guessing the pattern. The codebreakers try to guess the pattern, in
both order and number, within twelve turns. Each guess is made by writing
a row of four .
4. Provide feedback. After each guess the codemaker provides feedback by
placing from zero to four key dots next to the guess.
1. A full key dot [●] indicates the existence of a code number which is correct in both number and
position.
2. An empty dot [○] indicates the existence of a correct number placed in the wrong position.
5. Keep making guesses. Once feedback is provided, another guess is made.
Guesses and feedback continue until either the codebreakers guess
correctly, or twelve incorrect guesses are made.
INSTRUCTIONS
@mattphilip #AgileMW2019
15. As before with the following additions:
• Before a guess. Write down the intent of the guess –
what it is you want to learn.
• After feedback. Write down the result and what you
actually learned.
WORKSHEET
@mattphilip #AgileMW2019
17. • Define a question
• Gather information and resources (observe)
• Form an explanatory hypothesis
• Test the hypothesis by performing and experiment
and collecting data in a replicable manner
• Analyze the data
• Interpret the data and draw conclusions that serve as
a starting point for a new hypothesis
• Publish results
SCIENTIFICMETHOD
@mattphilip #AgileMW2019
26. • Company policy to give a bonus to employees who
refer new hires
• Implementing SAFe
• Rolling out a new team-leadership model
• Work-from home policy
• [Your own example]
HOWWOULDYOUTESTTHESE?
@mattphilip #AgileMW2019
27. • Psychological safety to fail
• Learning is outcome
• Time-based
• Measurable
• Single variable
• Provable (and disprovable) hypothesis
ELEMENTSOFGOODEXPERIMENTS
@mattphilip #AgileMW2019
32. SOURCES AND RESOURCES
◆ Karl Scotland (https://availagility.co.uk/)
◆ https://yorkesoftware.com/2018/03/09/goldratt-user-stories/
◆ http://cognitive-edge.com/methods/safe-to-fail-probes/
◆ Eleusis Express http://www.logicmazes.com/games/eleusis/express.html
◆ Freakonomics Radio: 370. How to Fail like a Pro
◆ Unlearn by Barry O’Reilly
◆ Agile BOSSA nova book
◆ How to Measure Anything
◆ John Cutler
◆ The Surprising Power of Online Experiments (https://hbr.org/2017/09/the-surprising-power-
of-online-experiments)
◆ https://exp-platform.com/Documents/2015-
08OnlineControlledExperimentsKDDKeynoteNR.pdf
◆ http://www.theagilefactor.com/
◆ https://barryoreilly.com/how-to-implement-hypothesis-driven-development/
◆ http://mattphilip.wordpress.com
@mattphilip #AgileMW2019