Keynote At Smidig Conference 2012 in Oslo, Norway. All rights reserved.

1. ENABLING
INNOVATION
Using Lean Startup &
Innovation Accounting
LUKAS FITTL
@lfittl
spark59.com
Lean Startup is trademarked by Eric Ries and used with permission.
Business Model Canvas is created by Alex Osterwalder and licensed under CC-BY-SA.

2. About Myself
Founded 2 tech startups,
worked with many others
& Spark59

3. Innovation in an
Agile Environment

4. Waterfall
Risk
BU ILD
BU ILD
B UILD
Time
Release!

5. Agile / XP / SCRUM
Risk
BUILD BUILD BUILD BUILD
Time
Release! Release! Release! Release!

6. Are we making our
product
better or worse?

7. Are we efficient?
vs
Are we effective?

8. The Definition of Done
should include testing
customer behaviour

9. Validated Learning

10. IDEAS
LEARN BUILD
DATA PRODUCT
MEASURE
Experiment

11. “State, before you do the work,
this is what I believe will happen,
and when you do the the work
see whether it happens or not.”
- Jabe Bloom

12. Hypotheses
instead of
Requirements

13. Requirement:
Users can create 3D avatars.

14. Hypothesis:
We believe that users will create a 3D
avatar and spend more time on other
users profiles.

15. Falsifiable Hypothesis:
50% of newly signed up users will create a 3D avatar,
Avg time spent on user profiles will go from 30s to 1min+

16. Be closer to reality,
and avoid
bias & comfort zones

17. User Stories != Experiments
Missing the “Measure” Stage
No Way to Validate the Learning

18. Deploy != Launch
if $rollout.active?(:chat, current_user)
...
else
...
end

19. Deploy Every Commit
Release To A Small Set of Customers
Launch The New Feature When Ready

20. Experiments need to be
shown to actual customers
Not just to your team!

21. Tools & Ta
c tics
Lean Stack:
Experiment Reports

22. EXPERIMENT REPORT Title: [TITLE] Author: [NAME] Created: [DATE]
Background Results
What are you trying to learn or achieve? Enter the data.
Falsifiable Hypothesis
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]
Validated Learning VALIDATED or INVALIDATED
Summarize your learning from the experiment.
Experiment Scope
How many and/or how long will the experiment last?
Minimum Build Plan
List the build plan step by step to get your measurement and experiment.
Next Action
Whatʼs the next experiment?
Lean Stack by Spark59.com

23. EXPERIMENT REPORT Title: [TITLE] Author: [NAME] Created: [DATE]
Background Results
What are you trying to learn or achieve? Enter the data.
Falsifiable Hypothesis
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]
Validated Learning VALIDATED or INVALIDATED
Summarize your learning from the experiment.
Experiment Scope
How many and/or how long will the experiment last?
Minimum Build Plan
List the build plan step by step to get your measurement and experiment.
Next Action
Whatʼs the next experiment?
Lean Stack by Spark59.com

24. Background
What are you trying to learn or achieve?
Give Context.
Why is this relevant?

25. Falsifiable Hypothesis
Your list of statements on the expected outcome of
the experiment.
Use this format:
[Specific Repeatable Action] will
[Expected Measurable Outcome]
Quantify Your Goals

26. Experiment Scope
How long will the experiment last?
Minimum Build Plan
List the build plan step by step to get your
measurement and experiment.
Set Scope & Timebox

27. Planning an Experiment
1. Context - Why is this relevant?
2. Quantify Your Goals
3. Set Scope & Timebox

28. Test your
Value and your Growth
Hypothesis first

29. Validated Learning:
Define Hypothesis,
Build & Launch Experiment,
Learn from the Results

30. Concepts for making this work:
1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

31. Hierarchy
1. Product Ownership
2. Shared Understanding
Anarchy
3. Accountability
4. Scoping & Timeboxing
Lean Startup

32. 1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

33. Centralized Product Ownership
Cross-Functional Product
Customers
Team Owner

34. Distributed Product Ownership
Business Cross-Functional
Customers
Stakeholder Team

35. Product Ownership is about
Defining Customer Value,
not Aesthetics

36. Business Experiments
instead of ( just)
Development Tasks

37. Experiments are
Owned By
A Team Member

38. Draws Expert
Resources as needed
(e.g. through pairing)

39. 1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

40. Why?
are we doing
what we’re doing

41. Team needs to
Understand The Impact
of their work

42. “Shared understanding is about
getting everyone to walk in the
other person’s shoes a little.”
Bill Scott - UX at Netflix, PayPal
This includes the business owner’s shoes!

43. Are we making progress
towards our business goal?

44. Tools & Ta
c tics
Company-Wide
Metrics Dashboard

46. Actionable
Accessible
Auditable

47. Actionable
Tie specific repeatable actions to
observed results.

48. Accessible
Let everyone in the company
see and understand whats going on.

49. Auditable
Tie the numbers back to actual
customers, so you can verify the metric
and believe in it.

50. You need all three
ACTIONABLE
AUDITABLE ACCESSIBLE

51. Unit Economics
instead of
Gross Revenue

52. Dave McClure’s AARRR Metrics
ACQUISITION How do users find you?
ACTIVATION Do users have a reat first experience?
RETENTION Do users come back?
REVENUE How do you make money?
REFERRAL Do users tell others?

53. Value Metrics
How do users find you? ACQUISITION
Do users have a great first experience? ACTIVATION
Do users come back? RETENTION
How do you make money? REVENUE
Do users tell others? REFERRAL

54. Growth Metrics
How do users find you? ACQUISITION
Do users have a great first experience? ACTIVATION
Do users come back? RETENTION
How do you make money? REVENUE
Do users tell others? REFERRAL

55. Tools & Ta
c tics
Cross-Functional
Pairing

56. Tools & Ta
c tics
Weekly Strategy Meeting
Part 1: Ideation

57. Avoid Brainstorming.
It causes
Premature Convergence
Around One Idea.

58. By Daniel Cook
http://www.lostgarden.com/2010/08/visualizing-creative-process.html

59. Design Studio
by Will Evans
@semanticwill

60. Observe & Explain Ideate Discuss
To The Group Alone In the Group
Sketch &
Write down!

61. Review Experiments
Only In Weekly Meeting
Focus On Efficiency
In Daily Stand-ups

62. 1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

63. Tools & Ta
c tics
Lean Stack:
Validated Learning Board

64. Kan-ban Board
IDEAS Build Measure Learn
LEARN BUILD ✓
✘
DATA PRODUCT
MEASURE

65. Build Measure Learn
✓
✘

66. 1 experiment
per team member

67. Separate Task Board
for tracking the details
Swimlane for each Experiment

68. Tools & Ta
c tics
Weekly Strategy Meeting
Part 2: Review

69. Review Experiment Results
Challenge the Interpretation
Kill / Restart overdue Experiments

70. EXPERIMENT REPORT Title: [TITLE] Author: [NAME] Created: [DATE]
Background Results
What are you trying to learn or achieve? Enter the data.
Falsifiable Hypothesis
Your list of statements on the expected outcome of the experiment.
Use this format:
[Specific Repeatable Action] will [Expected Measurable Outcome]
Validated Learning VALIDATED or INVALIDATED
Summarize your learning from the experiment.
Experiment Scope
How many and/or how long will the experiment last?
Minimum Build Plan
List the build plan step by step to get your measurement and experiment.
Next Action
Whatʼs the next experiment?
Lean Stack by Spark59.com

71. Results
Enter the data.
Actual data you Measured,
based on your Hypothesis.

72. Validated Learning
Summarize your learning from the experiment.
+
Validated
Invalidated Why?
Inconclusive

73. Next Action
Whatʼs the next experiment?
What will you do next
based on your learning?

74. ✘
Failure is good.
Validated Learning is our
Measurement of Progress.

75. 1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

76. Timeboxing

77. Build Measure Learn
✓
✘

78. Experiment Scope
Build Measure Learn
✓
✘

79. Startups don’t starve,
they drown.
Open-ended learning is dangerous,
always timebox.

80. Recommended Timeboxes:
Interviews/Prototypes: 1-2 Weeks
Qualitative Tests: 1 Month
Quantitative Tests: 2-3 Months

81. Customer Interview
(Paper) Prototype
MVP
“The Perfect Product”
Inside-the-Building Comfort Zone

82. Get-out-of-the-Building Reality
Customer Interview
(Paper) Prototype
MVP
“The Perfect Product”

83. Justify the Cost of MVPs
with Proof

84. Understand Problem:
Explorative Customer Interviews
Define Solution:
User Testing With Prototypes
Validate Qualitatively:
Release MVP and hand-collect feedback
10s to 100s of customers
Verify Quantitatively:
Partial Rollout or Split Test and measure improvement
100s to 10,000s of customers

86. In Summary

87. 0. Validated Learning
1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing

88. Tools & Ta
c tics
Experiment Reports
Company-wide Dashboard
Validated Learning Board
Weekly Strategy Meeting

89. Thanks!
leanstack.com
lukas@spark59.com