SlideShare a Scribd company logo
Startup Science ⑥
"Making a successful startup is an Art.
Making a startup that doesn’t fail is a Science.”
Created by
Masa Tadokoro
Product-Market Fit
Deliver the MVP
to the first
customer
Clarify what to
learn with an
MVP
Figure out things to
learn using the MVP
Chose MVP type
and build
Learn by delivering
to an evangelist customer
Conduct Qualitative analysis by
interviews with customers and
Quantitative analysis by
innovation metrics
Examine the UX
with Hooked model
Improve UX to
Enhance
the customer
engagement
Iterate product to
bring more value
to customers
Iterate product/ launch
it
Conduct Quantitative
& Qualitative Analysis
Build MVP
Repeat the
process until
the PMF has
been achieved
Create
Highly adaptive teamPivot the business model
Pivot the
Business Model
Copyright 2017 Masayuki Tadokoro All rights reserved
Copyright 2017 Masayuki Tadokoro All rights reserved
Build
Ideas
Product
Mea
sure
Data
Learn
Use measured data
for product iteration
What is a Lean Startup?
With a lean startup, you build the product for a
new idea or concept quickly (build MVP) and then
iterate on a build-measure-learn cycle based on
the customer's feedback of your product.
Your hypothesis is validated during the iteration
cycles.
By using the lean start-up method, start-up
founders will be able to find a product-market-fit
before burning up all of the resources.
”Discover qualitatively,
prove quantitively”
- Alistair Croll
Author of “Lean Analytics"
Copyright 2017 Masayuki Tadokoro All rights reserved
If you can’t measure it,
you can’t improve it.
P.F. Drucker
Question
❓
What to measure at the
early stage of a startup?
Copyright 2017 Masayuki Tadokoro All rights reserved
AARRR!
AARRR is a framework
to put users in a funnel
from acquisition to revenue
Acquisition: User acquisition
Activation: User sign-up
Retention: User comes back
Revenue: User pays for service
Referral: User introduces
other users
Introducing AARRR, pirate metrics.
Revenues increase as much as the
resources put into marketing.
5 times marketing costs leads to 5 times
customer acquisitions.
Copyright 2017 Masayuki Tadokoro All rights reserved
Before achieving product-market fit
(A state where you can create products what
customers need), customer acquisition by word of
mouth doesn’t work well; on the contrary, it will
lead a negative result.
Don’t stick to referral or customer
acquisition before achieving PMF.
Copyright 2017 Masayuki Tadokoro All rights reserved
Story Kanban Board
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Measure
Activation/
Retention/
Revenue
KGI
Acquisition Activation Retention Revenue
KPI Number of UU
accessing to the
sign up screen
Leaving rates
in the menu
screen
Percentage of
users starting
watching ads
Ad watch
completion
rates
Rates of
starting
using the
app
Factorize KGIs in each stage to actionable
KPIs
Leaving
rates in the
log in screen
Leaving
rates in the
menu
screen
Average number of ad
views per user
Sign up rates
Copyright 2017 Masayuki Tadokoro All rights reserved
Log in rates in 3days
after registering
Average number of ad
views per user
Percentage of users
who completed the
sequence
Sign up rates
Number of UU
accessing to the
sign up screen
First
Sign up
screen
Menu
screen
Start
watching
ad
Complete
watching
Wifi
usage
confirmation
Start
using wifi
Re-log in
screen
Menu
screen
Start
watching
ad
Complete
watching
KGI
Acquisition Activation Retention Revenue
KPI Number of UU
accessing to the
sign up screen
Leaving rates
in the menu
screen
Percentage of
users starting
watching ads
Ad watch
completion
rates
Rates of
starting
using the
app
Factorize KGIs in each stage to actionable
KPIs
Leaving
rates in the
log in screen
Leaving
rates in the
menu
screen
Average number of ad
views per user
Sign up rates
Copyright 2017 Masayuki Tadokoro All rights reserved
Log in rates in 3days
after registering
Average number of ad
views per user
Percentage of users
who completed the
sequence
Sign up rates
Number of UU
accessing to the
sign up screen
First
Sign up
screen
Menu
screen
Start
watching
ad
Complete
watching
Wifi
usage
confirmation
Start
using wifi
Re-log in
screen
Menu
screen
Start
watching
ad
Complete
watching
Result 335 ppl 302 ppl 93 ppl 23 ppl
4.2 times
( Total 96 tims)
Result of
measurement of
MVP (actual
number)
Result of
measurement of
MVP (rates)
Acquisition
(Visitors)
332 ppl 100%
Activation
(Registers) 305 ppl 90%
Activation
(completed
sequence)
93 ppl 30%
Retention
(Revisit in 3 days) 23 ppl 25%
Revenue
(Average ad views in a
day per user)
4.2 times 4.2 times
Show results quantitively using rates.
describe
the rates
Copyright 2017 Masayuki Tadokoro All rights reserved
Why is measurement needed?
Proper recognition
where you are heading
Recognizing a gap
between reality and goal may lead action
items to fill in the gap
KPIs will be
common language
among stakeholders
Copyright 2017 Masayuki Tadokoro All rights reserved
Recognizing a gap between reality and goal,
you may lead action items to fill in the gap
Copyright 2017 Masayuki Tadokoro All rights reserved
Result of
measurement of
MVP (actual
number)
Result of
measurement of
MVP (rates)
Goal
Acquisition
(Visitors)
332 ppl 100% 100%
Activation
(Registers) 305 ppl 90% 95%
Activation
(completed
sequence)
93 ppl 30% 80%
Retention
(Revisit in 3 days) 23 ppl 25% 80%
Revenue
(Average ad views in a
day per user)
4.2 times 4.2 times 10 times
Object to
achieve
What is a good indicator?
A good indicator
is measurable
A good indicator
Is easy to see
A good indicator
is actionable
A good indicator
is an indicator
of cause & effect
(An indicator
occurs changes
in other indicators)
Copyright 2017 Masayuki Tadokoro All rights reserved
Traps you may fall into
when setting KPIs.
See only result indicators:
UU, PV, revenue, CPA,
Etc.
See correlation indicators.
and repeater rates,
You mistakenly see a correlation,
you may not come up with the idea not
cause and effect relationships of what to do
next
See unactionable
indicators:
Indicating rough new user
rates
Copyright 2017 Masayuki Tadokoro All rights reserved
You won’t be able to set perfect KPIs for
your startup at the beginning.
You will continuously verify
whether you are measuring
appropriate KPIs at each stage.
Throw Vanity Metrics Away
Page view: Number of visitors to the page;
it is a vanity metric if you do not have
ad-sense or banner ads.
Number of visitors: It is a vanity metric since you do not
know what types of people are coming how frequently.
Unique visitors: It is a vanity metric since you do not know why
users came and what they have done
Visit duration: You have to measure which duration
for specific pages. If customers are staying on FAQ or
support pages for a long time, it indicates that usability is bad
Sign-up number:
It is meaningless if you can not
verify open rate of e-mails
Follower, Fan, Like: It is meaningless if it is just a number.
Key point is how many people will take an action
when you post something on social media stream
Copyright 2017 Masayuki Tadokoro All rights reserved
So
What ?
Vainglory indicators show a
realistic state but it would say, "so
what?" and are un-actionable.
Copyright 2017 Masayuki Tadokoro All rights reserved
With vanity metrics,
you will be reassured,
but it won't provide you
a clear guidance for
what you should do next.
- Eric Ries
Lean Startup
Copyright 2017 Masayuki Tadokoro All rights reserved
But
the revenue is flat
When you
measure
page views,
it goes up to right
time
Page
View
page view
revenue
You can easily find a metric which
is “goes up to right”.
We tend to only see what we
want.
People live in wrapped reality.
Copyright 2017 Masayuki Tadokoro All rights reserved
What you want to learn
Can customers gain wifi data volume and utilize it?
Verified story to learn
A user activates the app and signs up.
He/she watches add and utilize gained data
volume
What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
Activation: Sign up rates, rates of completion of the sequence
Retention: Rates of re-logging in in 3 days after registration
Revenue: Average # of ad views per a day
20 ppl per a day/2 weeks
Activation: 30%
Retention:25%
Revenue: 4 times in a day
Listen to the story behind the
numbers.
Copyright 2017 Masayuki Tadokoro All rights reserved
Story Kanban Board
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Get customer
interviews
A user story is built on the user's subjective view
through quantitative data only, you may not get
any subjective insight.
Copyright 2017 Masayuki Tadokoro All rights reserved
Interview with customers who have
actually touched the MVP.
Copyright 2017 Masayuki Tadokoro All rights reserved
・Did you see any values on this product?
・What are top 3 features of value?
・Why did you feel that the features are valuable?
・What features didn’t you use and feel less valuable?
・Why these features them less valuable?
・Will you recommend this product to your family or
friends?
Question examples in an MVP interview.
Copyright 2017 Masayuki Tadokoro All rights reserved
Start-up Founder can speak directly to customers
while producing the product.
Competitive Advantage of Startups
Copyright 2017 Masayuki Tadokoro All rights reserved
Dig Out Insight from
The Customer
Copyright 2017 Masayuki Tadokoro All rights reserved
Customers’ viewpoint itself is superficial
or just an amateur analysis in many
cases.
Only listening to customers is
worthless.
Copyright 2017 Masayuki Tadokoro All rights reserved
カスタマーの声を
KJ法で定性的に検証
①インタビュー
データを集める
②データを
細かい単位に
分ける
③カードを
平面上に
展開して
グループ化する
④ひとまとめした
カード群にその
グループを適切に
表す表札をつける
⑤グループ相互の
関連性を最も
論理的に説明できる
ようカードを並べる
⑥カスタマーの成功
の本当の要因(真因)
を言語化する
カスタマーの成功
要因
要因
真因
Copyright 2017 Masayuki Tadokoro All rights reserved
カスタマーの言葉
インサイト
インサイト
最も深い
インサイト
カスタマーインサイトの検証
Copyright 2017 Masayuki Tadokoro All rights reserved
So what?
So what?
So what?
Interviewer: Thank you for sparing your time today.
Catherin (customer): You're welcome.
Interviewer: You're using our product this 1 week, so how is
that?
Catherin (customer): I'm satisfied with it. Gaining wifi data
volume by watching ads is really convenient.
Interviewer: Well, then, why do you feel it is convenient?
Catherin (customer): I love to watch Youtube anywhere,
anytime, and then I can add data volume anytime when the
data volume becomes less.
Customer Interview
Copyright 2017 Masayuki Tadokoro All rights reserved
Interviewer: I see. You lose data volume while watching
Youtube and then you'd like to get data volume?
Catherin (customer): Yes, watching Youtube for 5 minutes
consumes 10MB and then I'd watch ads and add 5MB
repeatedly.
Interviewer: Well, that's interesting. What do you think of the
process?
Catherin: Honestly, it is a little tiresome. I think it is easier to
use if customers can add data volume at one time.
Copyright 2017 Masayuki Tadokoro All rights reserved
Voice of customer
:Really convenient
Insight:can continually
add data volume
to watch movies
Insight: watching adds
to gain data volume
is tiresome
More convenient
if customers can gain
more amounts of
data volume one time
Example of Anywhere Online
So what?
So what?
So what?
Copyright 2017 Masayuki Tadokoro All rights reserved
What you want to learn
Can customers gain wifi data volume and utilize it?
Verified story to learn
A user activates the app and signs up.
He/she watches add and utilize gained data
volume
What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
Utilization is tiresome but customers use the
solution, feeling its value.
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
Activation: Sign up rates, rates of completion of
the sequence Retention: Rates of re-logging in
3 days after registration
Revenue: Average # of ad views per a day
20 ppl per a day/2 weeks
Activation: 30%
Retention:25%
Revenue: 4 times in a day
Copyright 2017 Masayuki Tadokoro All rights reserved
Is there CRM system,
accumulate it
It’s no good to give up an MVP quickly.
Hang up for a certain period
Attain an amount of qualitative data and
feedback.
Copyright 2017 Masayuki Tadokoro All rights reserved
What you want to learn
Can customers gain wifi data volume and utilize it?
Verified story to learn
A user activates the app and signs up.
He/she watches add and utilize gained data
volume
What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
Utilization is tiresome but customers use the
solution, feeling its value.
What you learned
Customers feel its value and use it but there is less motivation to stick to the product because of its
fewer amounts of gained data volume.
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
Activation: Sign up rates, rates of completion of the sequence
Retention: Rates of re-logging in in 3 days after registration
Revenue: Average # of ad views per a day
20 ppl per a day/2 weeks
Activation: 30%
Retention:25%
Revenue: 4 times in a day
Verbalize and verify what you learned
through MVP with your team
Your team
Copyright 2017 Masayuki Tadokoro All rights reserved
” What we learned through MVP”
What to Verify with your Team
・What the reason customers use the product?
・Which features customers see valuable? Why?
・Why don’t customers use the product?
・Where is right and wrong in product hypothesis we built?
・Is our imagined appraisal standards of the product are
overlapped somehow with those of customers? Where are
overlapped or not?
・What is the most precious lesson from MVP?
・Which existing features are needed improvement? Which
existing features needs to do away? What kinds of features are
needed to add on?
Copyright 2017 Masayuki Tadokoro All rights reserved
Put organized knowledge
creation in a process.
Tacit
Knowledge
Tacit
Knowledge
TacitTacit
Formal
Knowledge
Formal
Knowledge
FormalFormal
Indiv
idual
Indiv
idual
G
I
I
I
I
I
I
Indiv
idual
G
G G
G
Acquiring tacit
knowledge through
body, five senses,
and direct
Experience
」
Creation of
hypothesis
of concept by
interview,
policy, metaphor
Systematization of
knowledge based
on composition of
formal knowledge
Formalize formal
knowledge
Understand it as
a new tacit
knowledge
New knowledge (= insight) is invented in a spiral
which is created by subjective ‘tacit knowledge’
based on one’s experience and objective ‘formal
knowledge’ that is able to be verbalized that are
transformed from and to each other.
Yujiro Nonaka
Emeritus professor
of Hitotsubashi University
Copyright 2017 Masayuki Tadokoro All rights reserved
Qualitative feedback
from customers
Team
Quantitative data
Accumulate it for
company learning
Learning
accumulation
Copyright 2017 Masayuki Tadokoro All rights reserved
How to decide
the next step?
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Achieved product-market Fit?
Copyright 2017 Masayuki Tadokoro All rights reserved
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Without a PMF, the value of the product is not propagated to
customers by word of mouth and customers won't be
aggressive about using it. Any media would never applaud it
and selling cycles rotate slowly.
However, with a PMF, it's very clear.
Customers come to us to buy products as we produce.
Adding services, the number of usage increases and we
might hire more sales and customer support staff and
journalists will contact me.
- Marc Andreessen
Copyright 2017 Masayuki Tadokoro All rights reserved
Check form of standards of
PMF achievement.
・ Maintain high retention rates?
・ Is the process from customer acquisition to
increasing revenue regeneratable?
・ Would the business model generated increase the
revenue according to the LTV>CPA model?
・ Is it built in accordance with lean canvas?
Copyright 2017 Masayuki Tadokoro All rights reserved
Shawn Elise
If 40% of users answer that ‘it is painful for
me if the product is removed from the
market', the product will consecutively
gain customers in the market.
Copyright 2017 Masayuki Tadokoro All rights reserved
“Traction”
If Product-market fit is yet achieved,
think of the next strategy to render.
Product-Market Fit
Deliver the MVP
to the first
customer
Clarify what to
learn with an
MVP
Figure out things to
learn using the
MVP
Chose MVP type
and build
Learn by delivering
to an evangelist customer
Conduct Qualitative analysis by
interviews with customers and
Quantitative analysis by
innovation metrics
Examine the UX
with Hooked model
Improve UX to
Enhance
the customer
engagement
Iterate product to
bring more value
to customers
Iterate product/ launch
it
Conduct
Quantitative &
Qualitative Analysis
Build MVP
Repeat the
process until
the PMF has
been achieved
Create
Highly adaptive teamPivot the business model
Pivot the
Business Model
Copyright 2017 Masayuki Tadokoro All rights reserved
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Refer it in
deciding
next step
Business Model 1 Business Model 2 Business Model 3
Time
MVP Version2 Version4Version3 Version6Version5
Improve
Pivot Pivot
Render pivoting if
you realize the business model
can’t achieve PMF Improve Improve Improve Improve Improve Improve
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Render pivoting if you realize
the business model can’t
achieve PMF
Business Model 1 Business Model 2 Business Model 3
Time
MVP Version2 Version4Version3 Version6Version5
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Imp
rov
e
Imp
rov
e
Imp
rov
e
Deploy Deploy Deploy Deploy Deploy Deploy Deploy Deploy Deploy
Conduct deployment of
UX/function improvement
continually
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Conduct deployment of
UX/function improvement
continually
Business Model 1 Business Model 2 Business Model 3
Time
MVP Version2 Version4Version3 Version6Version5
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Impr
ove
Imp
rov
e
Imp
rov
e
Imp
rov
e
Version2 Version4Version3 Version5
Iteration Iteration Iteration Iteration Iteration
Rotate product iteration
based on what you learned
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Yes
Rotate product iteration
based on what you learned
Product-Market Fit
Deliver the MVP
to the first
customer
Clarify what to
learn with an
MVP
Figure out things to
learn using the
MVP
Chose MVP type
and build
Learn by delivering to an
evangelist customer
Conduct Qualitative analysis by
interviews with customers and
Quantitative analysis by
innovation metrics
Examine the UX
with Hooked model
Improve UX to
Enhance
the customer
engagement
Iterate product to
bring more value
to customers
Iterate product/ launch
it
Conduct
Quantitative &
Qualitative Analysis
Build MVP
Repeat the
process until
the PMF has
been achieved
Create
Highly adaptive teamPivot the business model
Pivot the
Business Model
Copyright 2017 Masayuki Tadokoro All rights reserved
Verify Product (MVP) Iteration
Is the existing product (MVP)
providing enough value?
Copyright 2017 Masayuki Tadokoro All rights reserved
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Can you provide enough value with
only existing product features?
It's better to brush up existing features than
to add new features.
Newly added features will be useless for users
in many cases.
Researchers say 20% to 80% of all features of a product are
used
Basically, be negative to adding features.
*Adding new features will increase hidden costs ( test,
adjustment, complexity, less attention ).
Copyright 2017 Masayuki Tadokoro All rights reserved
Copyright 2017 Masayuki Tadokoro All rights reserved
13%
7%
16%
19%
45%
Not used
Used anytime
Used often
used
sometimes
Used
occasionally
Frequently used features are 20% of
the whole.
Copyright 2017 Masayuki Tadokoro All rights reserved
Consider deleting features,
not adding.
“Kill the feature -
Is there any cool feature,
find and kill it”.
Dave McClure
Founder of 500 Startups
13%
7%
16%
19%
45%
Not used
Used anytime
Used often
used
sometimes
Used
occasionally
Copyright 2017 Masayuki Tadokoro All rights reserved
Delete unused features
A case where you consider
adding new features.
Copyright 2017 Masayuki Tadokoro All rights reserved
Consider an issue hypothesis to
verify in the next iteration.
Copyright 2017 Masayuki Tadokoro All rights reserved
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Add new
feature!
What you want to learn
Can customers gain wifi data volume and utilize it?
Verified story to learn
A user activates the app and signs up.
He/she watches add and utilize gained data
volume
What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
Utilization is tiresome but customers use the
solution, feeling its value.
What you learned
Customers feel its value and use it but there is less motivation to stick to the product because of its
fewer amounts of gained data volume.
What you want to learn from the next sprint
Increasing data volume customers can add one time will help customers feel less tiresome in using the
product
See whether customer retention rates are improved by it.
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
Activation: Sign up rates, rates of completion of the sequence
Retention: Rates of re-logging in in 3 days after registration
Revenue: Average # of ad views per a day
20 ppl per a day/2 weeks
Activation: 30%
Retention:25%
Revenue: 4 times in a day
What you want to learn by
adding new features
Issue hypothesis:
Customers currently can't gain wifi data
volume little by little.
The solution makes customers frustrated.
Copyright 2017 Masayuki Tadokoro All rights reserved
Iteration canvas of MVP Iteration canvas Version 2
Copyright 2017 Masayuki Tadokoro All rights reserved
Move the issue hypothesis you
want to learn/verify in the next
sprint on the new iteration
canvas.
What you want to learn
Increasing data volume customers can add one time will help customers feel
less tiresome in using the product
See whether customer retention rates are improved by it.
Verified story to learn What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
Copyright 2017 Masayuki Tadokoro All rights reserved
Verify new possible stories (add features).
Possible story 1: Users are able to watch ad movies
in the 3 consecutive times (currently, users need to
return to the top page each time they watch an ad )
Possible story 2: Users add more data volume by
watching an ad (currently 10MB to 15MB).
Possible story 3: Users add 30MB (3 times of by watching
an ad) by answering questionnaires.
Copyright 2017 Masayuki Tadokoro All rights reserved
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Put possible stories on
Story Kanban board
Story Kanban Board
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Story
2-2
Story
2-3
Story
2-1
Put potential
stories in the next
iteration
Story Kanban Board
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Story
2-2
Story
2-3
Story
Version 2
Extract story
Version2
What you want to learn
Increasing data volume customers can add one time will help customers feel
less tiresome in using the product
See whether customer retention rates are improved by it.
Verified story to learn What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
Users add 30MB (3 times of by watching an ad)
by answering questionnaires
Run the same cycle
again
Story Kanban Board
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Story
2-2
Story
2-3
Story
Version 2
Begin building
Story of Anywhere online
Version 2 Usage
confirmation
Confirm
Questio
nnaires
Confirm
screen
Ad watch Confirm
screen
Added story
Copyright 2017 Masayuki Tadokoro All rights reserved
Open screen Menu screen
A story to be verified:
A user activates the app and signs up.
He/she watches add and utilize gained data
volume
What you want to learn
Increasing data volume customers can add one time will help customers feel
less tiresome in using the product
See whether customer retention rates are improved by it.
Verified story to learn What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
Users add 30MB (3 times of by watching an ad)
by answering questionnaires
Activation: Sign up rates, rates of completion of
the sequence
Retention: Rates of re-logging
in 3 days after registration
Revenue: Average # of ad views per a day
5 ppl per a day/1 weeks
Wash out implement
image and estimated
production costs of
KPI/KGI
Story Kanban Board
Backlog In Progress Completed Validation
Backlog storyaa
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Story
2-2
Story
2-3
Story
Version 2
Build, launch and conduct
quantitative verification
KGI
Acquisition Activation Retention Revenue
KPI Number of UU
accessing to the
sign up screen
Leaving rates
in the menu
screen
Percentage of
users starting
watching ads
Ad watch
completion
rates
Rates of
starting
using the
app
Factorize KGIs in each stage to actionable
KPIs
Leaving
rates in the
log in screen
Leaving
rates in the
menu
screen
Average number of ad
views per user
Sign up rates
Factorization
Copyright 2017 Masayuki Tadokoro All rights reserved
Log in rates in 3days
after registering
Average number of ad
views per user
Percentage of users
who completed the
sequence
Sign up rates
Number of UU
accessing to the
sign up screen
First
Sign up
screen
Menu
screen
Start
watching
ad
Complete
watching
Wifi
usage
confirmation
Start
using wifi
Re-log in
screen
Menu
screen
Start
watching
ad
Complete
watching
Activation rates
People who
registered:
More than 90%
Activation rates
People who f
inished the sequence:
35%
Retention rates
People who re-logged in
after 3 days: 25%
Revenue
Times of viewing ads and
answering questionnaires:
6 times
Launch Anytime online Ver 2 and measured results.
Questio
nnaires
Confirm
screen
Questio
nnaires
Confirm
screen
Copyright 2017 Masayuki Tadokoro All rights reserved
First
Sign up
screen
Menu
screen
Start
watching
ad
Complete
watching
Wifi usage
confirmation
Start
using wifi
Re-log in
screen
Menu
screen
Start
watching
ad
Complete
watching
Make a cohort analysis report
based on split testing.
Copyright 2017 Masayuki Tadokoro All rights reserved
Split test:
A scientific experiment where
they allocate object people
into groups randomly and compare and measure actions
of each group (Control group A and variation group B).
*When grouping them, change more than 1 condition and measure it
Copyright 2017 Masayuki Tadokoro All rights reserved
Cohort analysis:
It visualizes whether your actions/
initiatives have been making a positive
impact on your business.
Then you can
discuss with the team what you should
do next.
Copyright 2017 Masayuki Tadokoro All rights reserved
Measurement
result of MVP
(actual number)
Measurement
result of MVP
(rates)
Measurement
result of Version 2
(actual number)
Measurement
result of Version 2
(rates)
Objective
rates
Acquisition
(visitors)
332 ppl 100% 325 ppl 100% 100%
Activation
(registration) 305 ppl 90% 291 ppl 90% 95%
Activation
(Sequence
completed)
93 ppl 30% 102 ppl 35% 80%
Retention
(revist in 3days) 23 ppl 25% 25 ppl 25% 80%
Revenue
Number of ad views per user
in a day)
4.2 times 4.2 times 6.1 times 6.1 times 10 times
MVPと
バージョン2の
比較を行う
Compare MVP and its
Ver. 2
Copyright 2017 Masayuki Tadokoro All rights reserved
Introducing split test/cohort report makes
short term work annoying somehow.
However, implementing a system that
values your work quantitatively will
help you shorten the time in the
long term.
Copyright 2017 Masayuki Tadokoro All rights reserved
What you want to learn
Increasing data volume customers can add one time will help customers feel
less tiresome in using the product
See whether customer retention rates are improved by it.
Verified story to learn What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
5 ppl per a day/1 weeks
Users add 30MB (3 times of by watching an ad)
by answering questionnaires
Activation: Sign up rates, rates of completion of
the sequence
Retention: Rates of re-logging
in 3 days after registration
Revenue: Average # of ad views per a day
Write down quantitative
results
Interview with customers who
have actually used version 2.
Copyright 2017 Masayuki Tadokoro All rights reserved
What you want to learn
Increasing data volume customers can add one time will help customers feel
less tiresome in using the product
See whether customer retention rates are improved by it.
Verified story to learn What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
5 ppl per a day/1 weeks
Write down quantitative
reusults
Users add 30MB (3 times of by watching an ad)
by answering questionnaires
Activation: Sign up rates, rates of completion of
the sequence
Retention: Rates of re-logging
in 3 days after registration
Revenue: Average # of ad views per a day
What you want to learn
Increasing data volume customers can add one time will help customers feel
less tiresome in using the product
See whether customer retention rates are improved by it.
Verified story to learn What KPIs do you measure to verify?
What you build Cost & time to build
Quantitative result Qualitative result
What you learned
What you want to learn from the next sprint
Iteration Canvas
Copyright 2017 Masayuki Tadokoro All rights reserved
5 ppl per a day/1 weeks
Write down what you
learned
Users add 30MB (3 times of by watching an ad)
by answering questionnaires
Activation: Sign up rates, rates of completion of
the sequence
Retention: Rates of re-logging
in 3 days after registration
Revenue: Average # of ad views per a day
The batch size of iteration tends to be bigger as
time passes
As the batch proceeds to next phase, addition, redos
and delays occurs.
To make the overheads smaller, the butch will get
bigger naturally.
Copyright 2017 Masayuki Tadokoro All rights reserved
Important Points
Story Kanban Board
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Story
2-2
Story
2-3
Story
To shorten time, renew 2
stories (features) to
make them one and
examine it
Story Story Story
Story
Story
Story
(Up to 3)(Up to 3) (Up to 3) (Up to 3) (Up to 3)
Story Kanban Board
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Story
2-2
Story
2-3
StoryStory Story Story
Story
Story
(Up to 3)(Up to 3) (Up to 3) (Up to 3) (Up to 3)
Make the batch size
bigger and verify at
one time
Backlog In Progress Completed Validation
Backlog story
Extract stories that
you want to learn in
the iteration
Build
Build
completed
launch
Quantitative
analysis (split
test, cohort
analysis)
Qualitative
analysis/introspection
by customer interview
MVP
story
Story
1-1
Story
1-2
Copyright 2017 Masayuki Tadokoro All rights reserved
Story
2-2
Story
2-3
StoryStory Story Story
Story
Story
(Up to 3)(Up to 3) (Up to 3) (Up to 3) (Up to 3)
Not sure which story
(function) is more
effective…
Story Kanban Board
Fall into the death spiral of the
big batch.
Making a batch size bigger will
keep you from achieving the objective
of learning something in each
iteration.
Copyright 2017 Masayuki Tadokoro All rights reserved
In case you add
a simple feature
that has minimal
production costs,
add it to the issue board.
Copyright 2017 Masayuki Tadokoro All rights reserved
No
Yes
Add it on Story Kanban board as
story
Launch MVP and conduct
qualitative and quantitative
analysis
Achieved PMF?
Can improve it if continuing to use
the same business model?
Conduct transition of the
organization to scale
Yes
Consider to pivot
No
Yes
No
Flow chart to decide the next step.
Yes
No
Add it on issue Kanban board as
feature
Takes amounts of
features/issues?
Yes
No
Takes an amounts of
features/issues?
Verify UX/function improvement of
existing features
Provide enough values with only
existing features?
Verify product iteration added
features
Copyright 2017 Masayuki Tadokoro All rights reserved
Add it on issue Kanban board as
feature
Add it on Story Kanban board as
story
Add it to the issue
Kanban board
if it has minimal
production costs
As a timing of implement is coming
up, adopt a just-in-time method where
you produce a needed feature when
it's needed based on an analysis to
verify which items are needed.
Copyright 2017 Masayuki Tadokoro All rights reserved
・You can analyze based on the latest and
relevant information (enforcing you to answer the
reason why you are doing it now)
・Developers can learn more from customers
・It may keep you from reworking
・You may verify qualities of each batch earlier
using split test/cohort analysis
The effect of the just-in-time
method
Copyright 2017 Masayuki Tadokoro All rights reserved
Product
Market-Fit
Risk
(= Risk not to
create needed
products)
Time
PMF risk is rising
without customer
Feedback
Reduce the risk that you could produce unneeded
products for which you have consumed a lot of
resources/time.
PMF risk is limited with
customer feedback
High
Low
Copyright 2017 Masayuki Tadokoro All rights reserved
How to rhythmically manage the
just-in-time method.
・Utilize issue Kanban board
・Conduct daily stand up
・Proceed development based on customer
interview
・Develop avoiding documentation
*Nevertheless, document customer insights to capitalize/share in company
Copyright 2017 Masayuki Tadokoro All rights reserved
Backlog In process Completion Validation
Backlog story Building
Build completed
launch
Quantitative analysis
(split test, cohort analysis)
Features
Features
Features
Put features
in order of priority
Copyright 2017 Masayuki Tadokoro All rights reserved
Utilize the Issue Kanban Board
Backlog In process Completion Validation
Backlog story Building
Build completed
launch
Quantitative analysis
(split test, cohort analysis)
Features
Features
Features
Copyright 2017 Masayuki Tadokoro All rights reserved
Set a limitation
similar to the Story
Kanban board
(Up to 3) (Up to 3) (Up to 3) (Up to 3)
Utilize the Issue Kanban Board
Backlog In process Completion Validation
Backlog story Building
Build completed
launch
Quantitative analysis
(split test, cohort analysis)
Features
Features
Features
Copyright 2017 Masayuki Tadokoro All rights reserved
(Up to 3) (Up to 3) (Up to 3) (Up to 3)
Feature
Feature
Feature Feature
Feature
Feature
In case there is a bottle-neck,
use resources and increase
throughput
Utilize the Issue Kanban Board
Share the development process in
daily stand up meetings.
3 things to share in a daily stand up:
①What you did yesterday
②What you will do today
③Troubles that may keep your team from
working efficiently
Copyright 2017 Masayuki Tadokoro All rights reserved
Stop documentation in developing.
You may try to work in accordance with the
document.
Copyright 2017 Masayuki Tadokoro All rights reserved
Qualitative feedback
from customers
Team
Quantitative data
Accumulate it for
company learning
Learning
accumulation
Copyright 2017 Masayuki Tadokoro All rights reserved
Document customer insights to share in your team.
Rotate product iteration with a
rhythm.
Copyright 2017 Masayuki Tadokoro All rights reserved

More Related Content

What's hot

Growth Marketing Conference '17 Atlanta - Creating a Company Wide Growth Culture
Growth Marketing Conference '17 Atlanta - Creating a Company Wide Growth CultureGrowth Marketing Conference '17 Atlanta - Creating a Company Wide Growth Culture
Growth Marketing Conference '17 Atlanta - Creating a Company Wide Growth CultureThibault Imbert
 
Lean Discovery - Scrum Australia 2016
Lean Discovery - Scrum Australia 2016Lean Discovery - Scrum Australia 2016
Lean Discovery - Scrum Australia 2016Chris Chan
 
Growth Hacking Conference '17 - Antwerp
Growth Hacking Conference '17 - AntwerpGrowth Hacking Conference '17 - Antwerp
Growth Hacking Conference '17 - AntwerpThibault Imbert
 
Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...
Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...
Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...Lean Startup Co.
 
Designing great products and the skinny on MVP, Minimum Viable Products
Designing great products and the skinny on MVP, Minimum Viable ProductsDesigning great products and the skinny on MVP, Minimum Viable Products
Designing great products and the skinny on MVP, Minimum Viable ProductsAliza Carpio
 
14 Tips to Entrepreneurs to start the Right Stuff
14 Tips to Entrepreneurs to start the Right Stuff14 Tips to Entrepreneurs to start the Right Stuff
14 Tips to Entrepreneurs to start the Right StuffPatrick Stähler
 
The Care and Feeding of Ideas
The Care and Feeding of IdeasThe Care and Feeding of Ideas
The Care and Feeding of Ideas_gmathur
 
Building your minimum viable product
Building your minimum viable productBuilding your minimum viable product
Building your minimum viable productdarrylfriar
 
Navigabiz - Strategies for startup success
Navigabiz - Strategies for startup successNavigabiz - Strategies for startup success
Navigabiz - Strategies for startup successJeff Safovich
 
Smart and Scrappy Customer Development and Validation
Smart and Scrappy Customer Development and ValidationSmart and Scrappy Customer Development and Validation
Smart and Scrappy Customer Development and ValidationThe Garage Group
 
Gibson biddle- Productized Masterclasses
Gibson biddle- Productized Masterclasses Gibson biddle- Productized Masterclasses
Gibson biddle- Productized Masterclasses Productized
 
How to Build a Billion Dollar Startup
How to Build a Billion Dollar StartupHow to Build a Billion Dollar Startup
How to Build a Billion Dollar StartupJeff Safovich
 
Don’t be Uncle Rico via ProductCamp Cincinnati 2014
Don’t be Uncle Rico via ProductCamp Cincinnati 2014Don’t be Uncle Rico via ProductCamp Cincinnati 2014
Don’t be Uncle Rico via ProductCamp Cincinnati 2014The Garage Group
 
Adopting an MVP Perspective
Adopting an MVP PerspectiveAdopting an MVP Perspective
Adopting an MVP PerspectiveThe Garage Group
 
Intro to Lean Startup - Women's Startup Lab April 2015
Intro to Lean Startup - Women's Startup Lab April 2015Intro to Lean Startup - Women's Startup Lab April 2015
Intro to Lean Startup - Women's Startup Lab April 2015Kevin Shutta
 
Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...
Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...
Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...BSGAfrica
 
How to build a startup
How to build a startupHow to build a startup
How to build a startupRaomal Perera
 

What's hot (20)

Growth Marketing Conference '17 Atlanta - Creating a Company Wide Growth Culture
Growth Marketing Conference '17 Atlanta - Creating a Company Wide Growth CultureGrowth Marketing Conference '17 Atlanta - Creating a Company Wide Growth Culture
Growth Marketing Conference '17 Atlanta - Creating a Company Wide Growth Culture
 
Illuminate 2017
Illuminate 2017Illuminate 2017
Illuminate 2017
 
Lean Discovery - Scrum Australia 2016
Lean Discovery - Scrum Australia 2016Lean Discovery - Scrum Australia 2016
Lean Discovery - Scrum Australia 2016
 
Growth Hacking Conference '17 - Antwerp
Growth Hacking Conference '17 - AntwerpGrowth Hacking Conference '17 - Antwerp
Growth Hacking Conference '17 - Antwerp
 
The Lean Hardware Toolbox
The Lean Hardware ToolboxThe Lean Hardware Toolbox
The Lean Hardware Toolbox
 
Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...
Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...
Lean Impact–Lean Startup for Mission-driven Organizations by Leanne Pittsford...
 
Designing great products and the skinny on MVP, Minimum Viable Products
Designing great products and the skinny on MVP, Minimum Viable ProductsDesigning great products and the skinny on MVP, Minimum Viable Products
Designing great products and the skinny on MVP, Minimum Viable Products
 
Metrics that matter
Metrics that matterMetrics that matter
Metrics that matter
 
14 Tips to Entrepreneurs to start the Right Stuff
14 Tips to Entrepreneurs to start the Right Stuff14 Tips to Entrepreneurs to start the Right Stuff
14 Tips to Entrepreneurs to start the Right Stuff
 
The Care and Feeding of Ideas
The Care and Feeding of IdeasThe Care and Feeding of Ideas
The Care and Feeding of Ideas
 
Building your minimum viable product
Building your minimum viable productBuilding your minimum viable product
Building your minimum viable product
 
Navigabiz - Strategies for startup success
Navigabiz - Strategies for startup successNavigabiz - Strategies for startup success
Navigabiz - Strategies for startup success
 
Smart and Scrappy Customer Development and Validation
Smart and Scrappy Customer Development and ValidationSmart and Scrappy Customer Development and Validation
Smart and Scrappy Customer Development and Validation
 
Gibson biddle- Productized Masterclasses
Gibson biddle- Productized Masterclasses Gibson biddle- Productized Masterclasses
Gibson biddle- Productized Masterclasses
 
How to Build a Billion Dollar Startup
How to Build a Billion Dollar StartupHow to Build a Billion Dollar Startup
How to Build a Billion Dollar Startup
 
Don’t be Uncle Rico via ProductCamp Cincinnati 2014
Don’t be Uncle Rico via ProductCamp Cincinnati 2014Don’t be Uncle Rico via ProductCamp Cincinnati 2014
Don’t be Uncle Rico via ProductCamp Cincinnati 2014
 
Adopting an MVP Perspective
Adopting an MVP PerspectiveAdopting an MVP Perspective
Adopting an MVP Perspective
 
Intro to Lean Startup - Women's Startup Lab April 2015
Intro to Lean Startup - Women's Startup Lab April 2015Intro to Lean Startup - Women's Startup Lab April 2015
Intro to Lean Startup - Women's Startup Lab April 2015
 
Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...
Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...
Stop Guessing BSG Market Segmentation and Product Analysis Conference (Septem...
 
How to build a startup
How to build a startupHow to build a startup
How to build a startup
 

Similar to Startup Science 2017 ⑥

Data analytics-training - Jess Le Merle
Data analytics-training - Jess Le MerleData analytics-training - Jess Le Merle
Data analytics-training - Jess Le MerleHayden Armour
 
Applying the science of measurement to the art of advertising - 20 january 2...
Applying the science of measurement to the art of advertising  - 20 january 2...Applying the science of measurement to the art of advertising  - 20 january 2...
Applying the science of measurement to the art of advertising - 20 january 2...Ron Jacobs
 
Growth Hacking - 10 Key Checklist
Growth Hacking - 10 Key Checklist Growth Hacking - 10 Key Checklist
Growth Hacking - 10 Key Checklist Bryan Ferguson
 
Digital marketing metrics and measurement
Digital marketing metrics and measurementDigital marketing metrics and measurement
Digital marketing metrics and measurementsemrush_webinars
 
Qualityo: capabilities overview
Qualityo: capabilities overviewQualityo: capabilities overview
Qualityo: capabilities overviewQualityo
 
Customer, market and business validation for early-stage startups
Customer, market and business validation for early-stage startupsCustomer, market and business validation for early-stage startups
Customer, market and business validation for early-stage startupsJeff McClelland
 
Click Ventures Startup Metrics Playbook by Summer Interns 2018
Click Ventures Startup Metrics Playbook by Summer Interns 2018Click Ventures Startup Metrics Playbook by Summer Interns 2018
Click Ventures Startup Metrics Playbook by Summer Interns 2018Frederick Ng
 
Zodpovědně na automatizovaný účet
Zodpovědně na automatizovaný účetZodpovědně na automatizovaný účet
Zodpovědně na automatizovaný účetTaste Medio
 
Buyer Journey Workshop (Marketing Attribution)
Buyer Journey Workshop (Marketing Attribution)Buyer Journey Workshop (Marketing Attribution)
Buyer Journey Workshop (Marketing Attribution)Kashif Khurshid
 
A guide to Digital attribution models
A guide to Digital attribution modelsA guide to Digital attribution models
A guide to Digital attribution modelsMatthew Robinson
 
Digital Marketing Framework & Hacking Beyond It : Saptarshi Roy Chaudhury
Digital Marketing Framework & Hacking Beyond It : Saptarshi Roy ChaudhuryDigital Marketing Framework & Hacking Beyond It : Saptarshi Roy Chaudhury
Digital Marketing Framework & Hacking Beyond It : Saptarshi Roy ChaudhurySaptarshi Roy Chaudhury
 
Driving customer obsessed agenda
Driving customer obsessed agendaDriving customer obsessed agenda
Driving customer obsessed agendaAnastasia Pavlova
 
10 Metrics Every SaaS PM Should Use by fmr Facebook Product Leader
10 Metrics Every SaaS PM Should Use by fmr Facebook Product Leader10 Metrics Every SaaS PM Should Use by fmr Facebook Product Leader
10 Metrics Every SaaS PM Should Use by fmr Facebook Product LeaderProduct School
 
Analytics Academy 2017 Presentation Slides
Analytics Academy 2017 Presentation SlidesAnalytics Academy 2017 Presentation Slides
Analytics Academy 2017 Presentation SlidesHarvardComms
 

Similar to Startup Science 2017 ⑥ (20)

Multi-Channel Attribution - Where do Leads Come From?
Multi-Channel Attribution - Where do Leads Come From?Multi-Channel Attribution - Where do Leads Come From?
Multi-Channel Attribution - Where do Leads Come From?
 
Digital measurement & analytics
Digital measurement & analyticsDigital measurement & analytics
Digital measurement & analytics
 
Dgr sps16 act-on-final-deck
Dgr sps16 act-on-final-deckDgr sps16 act-on-final-deck
Dgr sps16 act-on-final-deck
 
Data analytics-training - Jess Le Merle
Data analytics-training - Jess Le MerleData analytics-training - Jess Le Merle
Data analytics-training - Jess Le Merle
 
Applying the science of measurement to the art of advertising - 20 january 2...
Applying the science of measurement to the art of advertising  - 20 january 2...Applying the science of measurement to the art of advertising  - 20 january 2...
Applying the science of measurement to the art of advertising - 20 january 2...
 
Bigdata Applications in Real Estate Marketing & Sales Strategies
Bigdata Applications in Real Estate Marketing & Sales StrategiesBigdata Applications in Real Estate Marketing & Sales Strategies
Bigdata Applications in Real Estate Marketing & Sales Strategies
 
Growth Hacking - 10 Key Checklist
Growth Hacking - 10 Key Checklist Growth Hacking - 10 Key Checklist
Growth Hacking - 10 Key Checklist
 
Digital marketing metrics and measurement
Digital marketing metrics and measurementDigital marketing metrics and measurement
Digital marketing metrics and measurement
 
Qualityo: capabilities overview
Qualityo: capabilities overviewQualityo: capabilities overview
Qualityo: capabilities overview
 
Customer, market and business validation for early-stage startups
Customer, market and business validation for early-stage startupsCustomer, market and business validation for early-stage startups
Customer, market and business validation for early-stage startups
 
Click Ventures Startup Metrics Playbook by Summer Interns 2018
Click Ventures Startup Metrics Playbook by Summer Interns 2018Click Ventures Startup Metrics Playbook by Summer Interns 2018
Click Ventures Startup Metrics Playbook by Summer Interns 2018
 
Zodpovědně na automatizovaný účet
Zodpovědně na automatizovaný účetZodpovědně na automatizovaný účet
Zodpovědně na automatizovaný účet
 
Real time marketing
Real time marketingReal time marketing
Real time marketing
 
Buyer Journey Workshop (Marketing Attribution)
Buyer Journey Workshop (Marketing Attribution)Buyer Journey Workshop (Marketing Attribution)
Buyer Journey Workshop (Marketing Attribution)
 
A guide to Digital attribution models
A guide to Digital attribution modelsA guide to Digital attribution models
A guide to Digital attribution models
 
Digital Marketing Framework & Hacking Beyond It : Saptarshi Roy Chaudhury
Digital Marketing Framework & Hacking Beyond It : Saptarshi Roy ChaudhuryDigital Marketing Framework & Hacking Beyond It : Saptarshi Roy Chaudhury
Digital Marketing Framework & Hacking Beyond It : Saptarshi Roy Chaudhury
 
CPMN - Product Led Growth (from Product Heads)
CPMN - Product Led Growth (from Product Heads)CPMN - Product Led Growth (from Product Heads)
CPMN - Product Led Growth (from Product Heads)
 
Driving customer obsessed agenda
Driving customer obsessed agendaDriving customer obsessed agenda
Driving customer obsessed agenda
 
10 Metrics Every SaaS PM Should Use by fmr Facebook Product Leader
10 Metrics Every SaaS PM Should Use by fmr Facebook Product Leader10 Metrics Every SaaS PM Should Use by fmr Facebook Product Leader
10 Metrics Every SaaS PM Should Use by fmr Facebook Product Leader
 
Analytics Academy 2017 Presentation Slides
Analytics Academy 2017 Presentation SlidesAnalytics Academy 2017 Presentation Slides
Analytics Academy 2017 Presentation Slides
 

More from Masa Tadokoro (20)

ストーリーでわかる「起業の科学」ハマりやすい57のワナスライド sd用
ストーリーでわかる「起業の科学」ハマりやすい57のワナスライド sd用ストーリーでわかる「起業の科学」ハマりやすい57のワナスライド sd用
ストーリーでわかる「起業の科学」ハマりやすい57のワナスライド sd用
 
MVV1
MVV1MVV1
MVV1
 
資金調達の極意②
資金調達の極意②資金調達の極意②
資金調達の極意②
 
資金調達の極意①
資金調達の極意①資金調達の極意①
資金調達の極意①
 
Open innovation 2018_0820 2
Open innovation 2018_0820 2Open innovation 2018_0820 2
Open innovation 2018_0820 2
 
Open innovation 2018_0820_1
Open innovation 2018_0820_1Open innovation 2018_0820_1
Open innovation 2018_0820_1
 
Open innovation 2018_0820_2
Open innovation 2018_0820_2Open innovation 2018_0820_2
Open innovation 2018_0820_2
 
Pivotの極意
Pivotの極意Pivotの極意
Pivotの極意
 
Fit8
Fit8Fit8
Fit8
 
Fit24
Fit24Fit24
Fit24
 
Fit26
Fit26Fit26
Fit26
 
Fit4
Fit4 Fit4
Fit4
 
Fit28
Fit28 Fit28
Fit28
 
Fit17
Fit17Fit17
Fit17
 
Fit23
Fit23Fit23
Fit23
 
Fit16
Fit16 Fit16
Fit16
 
Fit6
Fit6 Fit6
Fit6
 
Fit27
Fit27 Fit27
Fit27
 
Fit18
Fit18 Fit18
Fit18
 
Fit15
Fit15 Fit15
Fit15
 

Recently uploaded

Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...Björn Rohles
 
chapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxationchapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxationAUDIJEAngelo
 
Global Interconnection Group Joint Venture[960] (1).pdf
Global Interconnection Group Joint Venture[960] (1).pdfGlobal Interconnection Group Joint Venture[960] (1).pdf
Global Interconnection Group Joint Venture[960] (1).pdfHenry Tapper
 
State of D2C in India: A Logistics Update
State of D2C in India: A Logistics UpdateState of D2C in India: A Logistics Update
State of D2C in India: A Logistics UpdateRedSeer
 
Falcon Invoice Discounting Setup for Small Businesses
Falcon Invoice Discounting Setup for Small BusinessesFalcon Invoice Discounting Setup for Small Businesses
Falcon Invoice Discounting Setup for Small BusinessesFalcon investment
 
Business Valuation Principles for Entrepreneurs
Business Valuation Principles for EntrepreneursBusiness Valuation Principles for Entrepreneurs
Business Valuation Principles for EntrepreneursBen Wann
 
TriStar Gold Corporate Presentation May 2024
TriStar Gold Corporate Presentation May 2024TriStar Gold Corporate Presentation May 2024
TriStar Gold Corporate Presentation May 2024Adnet Communications
 
Did Paul Haggis Ever Win an Oscar for Best Filmmaker
Did Paul Haggis Ever Win an Oscar for Best FilmmakerDid Paul Haggis Ever Win an Oscar for Best Filmmaker
Did Paul Haggis Ever Win an Oscar for Best Filmmakerstajohn447
 
Pitch Deck Teardown: RAW Dating App's $3M Angel deck
Pitch Deck Teardown: RAW Dating App's $3M Angel deckPitch Deck Teardown: RAW Dating App's $3M Angel deck
Pitch Deck Teardown: RAW Dating App's $3M Angel deckHajeJanKamps
 
USA classified ads posting – best classified sites in usa.pdf
USA classified ads posting – best classified sites in usa.pdfUSA classified ads posting – best classified sites in usa.pdf
USA classified ads posting – best classified sites in usa.pdfsuperbizness1227
 
Hyundai capital 2024 1quarter Earnings release
Hyundai capital 2024 1quarter Earnings releaseHyundai capital 2024 1quarter Earnings release
Hyundai capital 2024 1quarter Earnings releaseirhcs
 
India’s Recommended Women Surgeons to Watch in 2024.pdf
India’s Recommended Women Surgeons to Watch in 2024.pdfIndia’s Recommended Women Surgeons to Watch in 2024.pdf
India’s Recommended Women Surgeons to Watch in 2024.pdfCIOLOOKIndia
 
IPTV Subscription UK: Your Guide to Choosing the Best Service
IPTV Subscription UK: Your Guide to Choosing the Best ServiceIPTV Subscription UK: Your Guide to Choosing the Best Service
IPTV Subscription UK: Your Guide to Choosing the Best ServiceDragon Dream Bar
 
falcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-indiafalcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-indiaFalcon Invoice Discounting
 
sales plan presentation by mckinsey alum
sales plan presentation by mckinsey alumsales plan presentation by mckinsey alum
sales plan presentation by mckinsey alumzyqmx62fgm
 
anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanasabutalha2013
 
12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdf
12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdf12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdf
12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdfSOFTTECHHUB
 
HR and Employment law update: May 2024.
HR and Employment law update:  May 2024.HR and Employment law update:  May 2024.
HR and Employment law update: May 2024.FelixPerez547899
 
Transforming Max Life Insurance with PMaps Job-Fit Assessments- Case Study
Transforming Max Life Insurance with PMaps Job-Fit Assessments- Case StudyTransforming Max Life Insurance with PMaps Job-Fit Assessments- Case Study
Transforming Max Life Insurance with PMaps Job-Fit Assessments- Case StudyPMaps Assessments
 
Special Purpose Vehicle (Purpose, Formation & examples)
Special Purpose Vehicle (Purpose, Formation & examples)Special Purpose Vehicle (Purpose, Formation & examples)
Special Purpose Vehicle (Purpose, Formation & examples)linciy03
 

Recently uploaded (20)

Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
Meaningful Technology for Humans: How Strategy Helps to Deliver Real Value fo...
 
chapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxationchapter 10 - excise tax of transfer and business taxation
chapter 10 - excise tax of transfer and business taxation
 
Global Interconnection Group Joint Venture[960] (1).pdf
Global Interconnection Group Joint Venture[960] (1).pdfGlobal Interconnection Group Joint Venture[960] (1).pdf
Global Interconnection Group Joint Venture[960] (1).pdf
 
State of D2C in India: A Logistics Update
State of D2C in India: A Logistics UpdateState of D2C in India: A Logistics Update
State of D2C in India: A Logistics Update
 
Falcon Invoice Discounting Setup for Small Businesses
Falcon Invoice Discounting Setup for Small BusinessesFalcon Invoice Discounting Setup for Small Businesses
Falcon Invoice Discounting Setup for Small Businesses
 
Business Valuation Principles for Entrepreneurs
Business Valuation Principles for EntrepreneursBusiness Valuation Principles for Entrepreneurs
Business Valuation Principles for Entrepreneurs
 
TriStar Gold Corporate Presentation May 2024
TriStar Gold Corporate Presentation May 2024TriStar Gold Corporate Presentation May 2024
TriStar Gold Corporate Presentation May 2024
 
Did Paul Haggis Ever Win an Oscar for Best Filmmaker
Did Paul Haggis Ever Win an Oscar for Best FilmmakerDid Paul Haggis Ever Win an Oscar for Best Filmmaker
Did Paul Haggis Ever Win an Oscar for Best Filmmaker
 
Pitch Deck Teardown: RAW Dating App's $3M Angel deck
Pitch Deck Teardown: RAW Dating App's $3M Angel deckPitch Deck Teardown: RAW Dating App's $3M Angel deck
Pitch Deck Teardown: RAW Dating App's $3M Angel deck
 
USA classified ads posting – best classified sites in usa.pdf
USA classified ads posting – best classified sites in usa.pdfUSA classified ads posting – best classified sites in usa.pdf
USA classified ads posting – best classified sites in usa.pdf
 
Hyundai capital 2024 1quarter Earnings release
Hyundai capital 2024 1quarter Earnings releaseHyundai capital 2024 1quarter Earnings release
Hyundai capital 2024 1quarter Earnings release
 
India’s Recommended Women Surgeons to Watch in 2024.pdf
India’s Recommended Women Surgeons to Watch in 2024.pdfIndia’s Recommended Women Surgeons to Watch in 2024.pdf
India’s Recommended Women Surgeons to Watch in 2024.pdf
 
IPTV Subscription UK: Your Guide to Choosing the Best Service
IPTV Subscription UK: Your Guide to Choosing the Best ServiceIPTV Subscription UK: Your Guide to Choosing the Best Service
IPTV Subscription UK: Your Guide to Choosing the Best Service
 
falcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-indiafalcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-india
 
sales plan presentation by mckinsey alum
sales plan presentation by mckinsey alumsales plan presentation by mckinsey alum
sales plan presentation by mckinsey alum
 
anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about venice
 
12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdf
12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdf12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdf
12 Conversion Rate Optimization Strategies for Ecommerce Websites.pdf
 
HR and Employment law update: May 2024.
HR and Employment law update:  May 2024.HR and Employment law update:  May 2024.
HR and Employment law update: May 2024.
 
Transforming Max Life Insurance with PMaps Job-Fit Assessments- Case Study
Transforming Max Life Insurance with PMaps Job-Fit Assessments- Case StudyTransforming Max Life Insurance with PMaps Job-Fit Assessments- Case Study
Transforming Max Life Insurance with PMaps Job-Fit Assessments- Case Study
 
Special Purpose Vehicle (Purpose, Formation & examples)
Special Purpose Vehicle (Purpose, Formation & examples)Special Purpose Vehicle (Purpose, Formation & examples)
Special Purpose Vehicle (Purpose, Formation & examples)
 

Startup Science 2017 ⑥

  • 1. Startup Science ⑥ "Making a successful startup is an Art. Making a startup that doesn’t fail is a Science.” Created by Masa Tadokoro
  • 2. Product-Market Fit Deliver the MVP to the first customer Clarify what to learn with an MVP Figure out things to learn using the MVP Chose MVP type and build Learn by delivering to an evangelist customer Conduct Qualitative analysis by interviews with customers and Quantitative analysis by innovation metrics Examine the UX with Hooked model Improve UX to Enhance the customer engagement Iterate product to bring more value to customers Iterate product/ launch it Conduct Quantitative & Qualitative Analysis Build MVP Repeat the process until the PMF has been achieved Create Highly adaptive teamPivot the business model Pivot the Business Model Copyright 2017 Masayuki Tadokoro All rights reserved
  • 3. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 4. Build Ideas Product Mea sure Data Learn Use measured data for product iteration What is a Lean Startup? With a lean startup, you build the product for a new idea or concept quickly (build MVP) and then iterate on a build-measure-learn cycle based on the customer's feedback of your product. Your hypothesis is validated during the iteration cycles. By using the lean start-up method, start-up founders will be able to find a product-market-fit before burning up all of the resources.
  • 5. ”Discover qualitatively, prove quantitively” - Alistair Croll Author of “Lean Analytics" Copyright 2017 Masayuki Tadokoro All rights reserved
  • 6. If you can’t measure it, you can’t improve it. P.F. Drucker
  • 7. Question ❓ What to measure at the early stage of a startup? Copyright 2017 Masayuki Tadokoro All rights reserved
  • 8. AARRR! AARRR is a framework to put users in a funnel from acquisition to revenue Acquisition: User acquisition Activation: User sign-up Retention: User comes back Revenue: User pays for service Referral: User introduces other users Introducing AARRR, pirate metrics.
  • 9. Revenues increase as much as the resources put into marketing. 5 times marketing costs leads to 5 times customer acquisitions. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 10. Before achieving product-market fit (A state where you can create products what customers need), customer acquisition by word of mouth doesn’t work well; on the contrary, it will lead a negative result. Don’t stick to referral or customer acquisition before achieving PMF. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 11. Story Kanban Board Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Measure Activation/ Retention/ Revenue
  • 12. KGI Acquisition Activation Retention Revenue KPI Number of UU accessing to the sign up screen Leaving rates in the menu screen Percentage of users starting watching ads Ad watch completion rates Rates of starting using the app Factorize KGIs in each stage to actionable KPIs Leaving rates in the log in screen Leaving rates in the menu screen Average number of ad views per user Sign up rates Copyright 2017 Masayuki Tadokoro All rights reserved Log in rates in 3days after registering Average number of ad views per user Percentage of users who completed the sequence Sign up rates Number of UU accessing to the sign up screen First Sign up screen Menu screen Start watching ad Complete watching Wifi usage confirmation Start using wifi Re-log in screen Menu screen Start watching ad Complete watching
  • 13. KGI Acquisition Activation Retention Revenue KPI Number of UU accessing to the sign up screen Leaving rates in the menu screen Percentage of users starting watching ads Ad watch completion rates Rates of starting using the app Factorize KGIs in each stage to actionable KPIs Leaving rates in the log in screen Leaving rates in the menu screen Average number of ad views per user Sign up rates Copyright 2017 Masayuki Tadokoro All rights reserved Log in rates in 3days after registering Average number of ad views per user Percentage of users who completed the sequence Sign up rates Number of UU accessing to the sign up screen First Sign up screen Menu screen Start watching ad Complete watching Wifi usage confirmation Start using wifi Re-log in screen Menu screen Start watching ad Complete watching Result 335 ppl 302 ppl 93 ppl 23 ppl 4.2 times ( Total 96 tims)
  • 14. Result of measurement of MVP (actual number) Result of measurement of MVP (rates) Acquisition (Visitors) 332 ppl 100% Activation (Registers) 305 ppl 90% Activation (completed sequence) 93 ppl 30% Retention (Revisit in 3 days) 23 ppl 25% Revenue (Average ad views in a day per user) 4.2 times 4.2 times Show results quantitively using rates. describe the rates Copyright 2017 Masayuki Tadokoro All rights reserved
  • 15. Why is measurement needed? Proper recognition where you are heading Recognizing a gap between reality and goal may lead action items to fill in the gap KPIs will be common language among stakeholders Copyright 2017 Masayuki Tadokoro All rights reserved
  • 16. Recognizing a gap between reality and goal, you may lead action items to fill in the gap Copyright 2017 Masayuki Tadokoro All rights reserved Result of measurement of MVP (actual number) Result of measurement of MVP (rates) Goal Acquisition (Visitors) 332 ppl 100% 100% Activation (Registers) 305 ppl 90% 95% Activation (completed sequence) 93 ppl 30% 80% Retention (Revisit in 3 days) 23 ppl 25% 80% Revenue (Average ad views in a day per user) 4.2 times 4.2 times 10 times Object to achieve
  • 17. What is a good indicator? A good indicator is measurable A good indicator Is easy to see A good indicator is actionable A good indicator is an indicator of cause & effect (An indicator occurs changes in other indicators) Copyright 2017 Masayuki Tadokoro All rights reserved
  • 18. Traps you may fall into when setting KPIs. See only result indicators: UU, PV, revenue, CPA, Etc. See correlation indicators. and repeater rates, You mistakenly see a correlation, you may not come up with the idea not cause and effect relationships of what to do next See unactionable indicators: Indicating rough new user rates Copyright 2017 Masayuki Tadokoro All rights reserved
  • 19. You won’t be able to set perfect KPIs for your startup at the beginning. You will continuously verify whether you are measuring appropriate KPIs at each stage.
  • 20. Throw Vanity Metrics Away Page view: Number of visitors to the page; it is a vanity metric if you do not have ad-sense or banner ads. Number of visitors: It is a vanity metric since you do not know what types of people are coming how frequently. Unique visitors: It is a vanity metric since you do not know why users came and what they have done Visit duration: You have to measure which duration for specific pages. If customers are staying on FAQ or support pages for a long time, it indicates that usability is bad Sign-up number: It is meaningless if you can not verify open rate of e-mails Follower, Fan, Like: It is meaningless if it is just a number. Key point is how many people will take an action when you post something on social media stream Copyright 2017 Masayuki Tadokoro All rights reserved
  • 21. So What ? Vainglory indicators show a realistic state but it would say, "so what?" and are un-actionable. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 22. With vanity metrics, you will be reassured, but it won't provide you a clear guidance for what you should do next. - Eric Ries Lean Startup Copyright 2017 Masayuki Tadokoro All rights reserved
  • 23. But the revenue is flat When you measure page views, it goes up to right time Page View page view revenue You can easily find a metric which is “goes up to right”.
  • 24. We tend to only see what we want. People live in wrapped reality. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 25. What you want to learn Can customers gain wifi data volume and utilize it? Verified story to learn A user activates the app and signs up. He/she watches add and utilize gained data volume What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in in 3 days after registration Revenue: Average # of ad views per a day 20 ppl per a day/2 weeks Activation: 30% Retention:25% Revenue: 4 times in a day
  • 26. Listen to the story behind the numbers. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 27. Story Kanban Board Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Get customer interviews
  • 28. A user story is built on the user's subjective view through quantitative data only, you may not get any subjective insight. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 29. Interview with customers who have actually touched the MVP. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 30. ・Did you see any values on this product? ・What are top 3 features of value? ・Why did you feel that the features are valuable? ・What features didn’t you use and feel less valuable? ・Why these features them less valuable? ・Will you recommend this product to your family or friends? Question examples in an MVP interview. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 31. Start-up Founder can speak directly to customers while producing the product. Competitive Advantage of Startups Copyright 2017 Masayuki Tadokoro All rights reserved
  • 32. Dig Out Insight from The Customer Copyright 2017 Masayuki Tadokoro All rights reserved
  • 33. Customers’ viewpoint itself is superficial or just an amateur analysis in many cases. Only listening to customers is worthless. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 36. Interviewer: Thank you for sparing your time today. Catherin (customer): You're welcome. Interviewer: You're using our product this 1 week, so how is that? Catherin (customer): I'm satisfied with it. Gaining wifi data volume by watching ads is really convenient. Interviewer: Well, then, why do you feel it is convenient? Catherin (customer): I love to watch Youtube anywhere, anytime, and then I can add data volume anytime when the data volume becomes less. Customer Interview Copyright 2017 Masayuki Tadokoro All rights reserved
  • 37. Interviewer: I see. You lose data volume while watching Youtube and then you'd like to get data volume? Catherin (customer): Yes, watching Youtube for 5 minutes consumes 10MB and then I'd watch ads and add 5MB repeatedly. Interviewer: Well, that's interesting. What do you think of the process? Catherin: Honestly, it is a little tiresome. I think it is easier to use if customers can add data volume at one time. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 38. Voice of customer :Really convenient Insight:can continually add data volume to watch movies Insight: watching adds to gain data volume is tiresome More convenient if customers can gain more amounts of data volume one time Example of Anywhere Online So what? So what? So what? Copyright 2017 Masayuki Tadokoro All rights reserved
  • 39. What you want to learn Can customers gain wifi data volume and utilize it? Verified story to learn A user activates the app and signs up. He/she watches add and utilize gained data volume What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result Utilization is tiresome but customers use the solution, feeling its value. What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in 3 days after registration Revenue: Average # of ad views per a day 20 ppl per a day/2 weeks Activation: 30% Retention:25% Revenue: 4 times in a day
  • 40. Copyright 2017 Masayuki Tadokoro All rights reserved Is there CRM system, accumulate it
  • 41. It’s no good to give up an MVP quickly. Hang up for a certain period Attain an amount of qualitative data and feedback. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 42. What you want to learn Can customers gain wifi data volume and utilize it? Verified story to learn A user activates the app and signs up. He/she watches add and utilize gained data volume What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result Utilization is tiresome but customers use the solution, feeling its value. What you learned Customers feel its value and use it but there is less motivation to stick to the product because of its fewer amounts of gained data volume. What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in in 3 days after registration Revenue: Average # of ad views per a day 20 ppl per a day/2 weeks Activation: 30% Retention:25% Revenue: 4 times in a day
  • 43. Verbalize and verify what you learned through MVP with your team Your team Copyright 2017 Masayuki Tadokoro All rights reserved ” What we learned through MVP”
  • 44. What to Verify with your Team ・What the reason customers use the product? ・Which features customers see valuable? Why? ・Why don’t customers use the product? ・Where is right and wrong in product hypothesis we built? ・Is our imagined appraisal standards of the product are overlapped somehow with those of customers? Where are overlapped or not? ・What is the most precious lesson from MVP? ・Which existing features are needed improvement? Which existing features needs to do away? What kinds of features are needed to add on? Copyright 2017 Masayuki Tadokoro All rights reserved
  • 45. Put organized knowledge creation in a process. Tacit Knowledge Tacit Knowledge TacitTacit Formal Knowledge Formal Knowledge FormalFormal Indiv idual Indiv idual G I I I I I I Indiv idual G G G G Acquiring tacit knowledge through body, five senses, and direct Experience 」 Creation of hypothesis of concept by interview, policy, metaphor Systematization of knowledge based on composition of formal knowledge Formalize formal knowledge Understand it as a new tacit knowledge
  • 46. New knowledge (= insight) is invented in a spiral which is created by subjective ‘tacit knowledge’ based on one’s experience and objective ‘formal knowledge’ that is able to be verbalized that are transformed from and to each other. Yujiro Nonaka Emeritus professor of Hitotsubashi University Copyright 2017 Masayuki Tadokoro All rights reserved
  • 47. Qualitative feedback from customers Team Quantitative data Accumulate it for company learning Learning accumulation Copyright 2017 Masayuki Tadokoro All rights reserved
  • 48. How to decide the next step?
  • 49. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story
  • 50. Achieved product-market Fit? Copyright 2017 Masayuki Tadokoro All rights reserved
  • 51. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story
  • 52. Without a PMF, the value of the product is not propagated to customers by word of mouth and customers won't be aggressive about using it. Any media would never applaud it and selling cycles rotate slowly. However, with a PMF, it's very clear. Customers come to us to buy products as we produce. Adding services, the number of usage increases and we might hire more sales and customer support staff and journalists will contact me. - Marc Andreessen Copyright 2017 Masayuki Tadokoro All rights reserved
  • 53. Check form of standards of PMF achievement. ・ Maintain high retention rates? ・ Is the process from customer acquisition to increasing revenue regeneratable? ・ Would the business model generated increase the revenue according to the LTV>CPA model? ・ Is it built in accordance with lean canvas? Copyright 2017 Masayuki Tadokoro All rights reserved
  • 54. Shawn Elise If 40% of users answer that ‘it is painful for me if the product is removed from the market', the product will consecutively gain customers in the market. Copyright 2017 Masayuki Tadokoro All rights reserved “Traction”
  • 55. If Product-market fit is yet achieved, think of the next strategy to render.
  • 56. Product-Market Fit Deliver the MVP to the first customer Clarify what to learn with an MVP Figure out things to learn using the MVP Chose MVP type and build Learn by delivering to an evangelist customer Conduct Qualitative analysis by interviews with customers and Quantitative analysis by innovation metrics Examine the UX with Hooked model Improve UX to Enhance the customer engagement Iterate product to bring more value to customers Iterate product/ launch it Conduct Quantitative & Qualitative Analysis Build MVP Repeat the process until the PMF has been achieved Create Highly adaptive teamPivot the business model Pivot the Business Model Copyright 2017 Masayuki Tadokoro All rights reserved
  • 57. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story Refer it in deciding next step
  • 58. Business Model 1 Business Model 2 Business Model 3 Time MVP Version2 Version4Version3 Version6Version5 Improve Pivot Pivot Render pivoting if you realize the business model can’t achieve PMF Improve Improve Improve Improve Improve Improve
  • 59. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story Render pivoting if you realize the business model can’t achieve PMF
  • 60. Business Model 1 Business Model 2 Business Model 3 Time MVP Version2 Version4Version3 Version6Version5 Impr ove Impr ove Impr ove Impr ove Impr ove Impr ove Impr ove Imp rov e Imp rov e Imp rov e Deploy Deploy Deploy Deploy Deploy Deploy Deploy Deploy Deploy Conduct deployment of UX/function improvement continually
  • 61. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story Conduct deployment of UX/function improvement continually
  • 62. Business Model 1 Business Model 2 Business Model 3 Time MVP Version2 Version4Version3 Version6Version5 Impr ove Impr ove Impr ove Impr ove Impr ove Impr ove Impr ove Imp rov e Imp rov e Imp rov e Version2 Version4Version3 Version5 Iteration Iteration Iteration Iteration Iteration Rotate product iteration based on what you learned
  • 63. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story Yes Rotate product iteration based on what you learned
  • 64. Product-Market Fit Deliver the MVP to the first customer Clarify what to learn with an MVP Figure out things to learn using the MVP Chose MVP type and build Learn by delivering to an evangelist customer Conduct Qualitative analysis by interviews with customers and Quantitative analysis by innovation metrics Examine the UX with Hooked model Improve UX to Enhance the customer engagement Iterate product to bring more value to customers Iterate product/ launch it Conduct Quantitative & Qualitative Analysis Build MVP Repeat the process until the PMF has been achieved Create Highly adaptive teamPivot the business model Pivot the Business Model Copyright 2017 Masayuki Tadokoro All rights reserved
  • 65. Verify Product (MVP) Iteration Is the existing product (MVP) providing enough value? Copyright 2017 Masayuki Tadokoro All rights reserved
  • 66. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story
  • 67. Can you provide enough value with only existing product features?
  • 68. It's better to brush up existing features than to add new features. Newly added features will be useless for users in many cases. Researchers say 20% to 80% of all features of a product are used Basically, be negative to adding features. *Adding new features will increase hidden costs ( test, adjustment, complexity, less attention ). Copyright 2017 Masayuki Tadokoro All rights reserved Copyright 2017 Masayuki Tadokoro All rights reserved
  • 69. 13% 7% 16% 19% 45% Not used Used anytime Used often used sometimes Used occasionally Frequently used features are 20% of the whole. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 71. “Kill the feature - Is there any cool feature, find and kill it”. Dave McClure Founder of 500 Startups
  • 72. 13% 7% 16% 19% 45% Not used Used anytime Used often used sometimes Used occasionally Copyright 2017 Masayuki Tadokoro All rights reserved Delete unused features
  • 73. A case where you consider adding new features. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 74. Consider an issue hypothesis to verify in the next iteration. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 75. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story Add new feature!
  • 76. What you want to learn Can customers gain wifi data volume and utilize it? Verified story to learn A user activates the app and signs up. He/she watches add and utilize gained data volume What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result Utilization is tiresome but customers use the solution, feeling its value. What you learned Customers feel its value and use it but there is less motivation to stick to the product because of its fewer amounts of gained data volume. What you want to learn from the next sprint Increasing data volume customers can add one time will help customers feel less tiresome in using the product See whether customer retention rates are improved by it. Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in in 3 days after registration Revenue: Average # of ad views per a day 20 ppl per a day/2 weeks Activation: 30% Retention:25% Revenue: 4 times in a day What you want to learn by adding new features
  • 77. Issue hypothesis: Customers currently can't gain wifi data volume little by little. The solution makes customers frustrated. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 78. Iteration canvas of MVP Iteration canvas Version 2 Copyright 2017 Masayuki Tadokoro All rights reserved Move the issue hypothesis you want to learn/verify in the next sprint on the new iteration canvas.
  • 79. What you want to learn Increasing data volume customers can add one time will help customers feel less tiresome in using the product See whether customer retention rates are improved by it. Verified story to learn What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved
  • 80. Copyright 2017 Masayuki Tadokoro All rights reserved Verify new possible stories (add features). Possible story 1: Users are able to watch ad movies in the 3 consecutive times (currently, users need to return to the top page each time they watch an ad ) Possible story 2: Users add more data volume by watching an ad (currently 10MB to 15MB). Possible story 3: Users add 30MB (3 times of by watching an ad) by answering questionnaires. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 81. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story Put possible stories on Story Kanban board
  • 82. Story Kanban Board Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Story 2-2 Story 2-3 Story 2-1 Put potential stories in the next iteration
  • 83. Story Kanban Board Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Story 2-2 Story 2-3 Story Version 2 Extract story Version2
  • 84. What you want to learn Increasing data volume customers can add one time will help customers feel less tiresome in using the product See whether customer retention rates are improved by it. Verified story to learn What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved Users add 30MB (3 times of by watching an ad) by answering questionnaires Run the same cycle again
  • 85. Story Kanban Board Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Story 2-2 Story 2-3 Story Version 2 Begin building
  • 86. Story of Anywhere online Version 2 Usage confirmation Confirm Questio nnaires Confirm screen Ad watch Confirm screen Added story Copyright 2017 Masayuki Tadokoro All rights reserved Open screen Menu screen A story to be verified: A user activates the app and signs up. He/she watches add and utilize gained data volume
  • 87. What you want to learn Increasing data volume customers can add one time will help customers feel less tiresome in using the product See whether customer retention rates are improved by it. Verified story to learn What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved Users add 30MB (3 times of by watching an ad) by answering questionnaires Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in 3 days after registration Revenue: Average # of ad views per a day 5 ppl per a day/1 weeks Wash out implement image and estimated production costs of KPI/KGI
  • 88. Story Kanban Board Backlog In Progress Completed Validation Backlog storyaa Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Story 2-2 Story 2-3 Story Version 2 Build, launch and conduct quantitative verification
  • 89. KGI Acquisition Activation Retention Revenue KPI Number of UU accessing to the sign up screen Leaving rates in the menu screen Percentage of users starting watching ads Ad watch completion rates Rates of starting using the app Factorize KGIs in each stage to actionable KPIs Leaving rates in the log in screen Leaving rates in the menu screen Average number of ad views per user Sign up rates Factorization Copyright 2017 Masayuki Tadokoro All rights reserved Log in rates in 3days after registering Average number of ad views per user Percentage of users who completed the sequence Sign up rates Number of UU accessing to the sign up screen First Sign up screen Menu screen Start watching ad Complete watching Wifi usage confirmation Start using wifi Re-log in screen Menu screen Start watching ad Complete watching
  • 90. Activation rates People who registered: More than 90% Activation rates People who f inished the sequence: 35% Retention rates People who re-logged in after 3 days: 25% Revenue Times of viewing ads and answering questionnaires: 6 times Launch Anytime online Ver 2 and measured results. Questio nnaires Confirm screen Questio nnaires Confirm screen Copyright 2017 Masayuki Tadokoro All rights reserved First Sign up screen Menu screen Start watching ad Complete watching Wifi usage confirmation Start using wifi Re-log in screen Menu screen Start watching ad Complete watching
  • 91. Make a cohort analysis report based on split testing. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 92. Split test: A scientific experiment where they allocate object people into groups randomly and compare and measure actions of each group (Control group A and variation group B). *When grouping them, change more than 1 condition and measure it Copyright 2017 Masayuki Tadokoro All rights reserved
  • 93. Cohort analysis: It visualizes whether your actions/ initiatives have been making a positive impact on your business. Then you can discuss with the team what you should do next. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 94. Measurement result of MVP (actual number) Measurement result of MVP (rates) Measurement result of Version 2 (actual number) Measurement result of Version 2 (rates) Objective rates Acquisition (visitors) 332 ppl 100% 325 ppl 100% 100% Activation (registration) 305 ppl 90% 291 ppl 90% 95% Activation (Sequence completed) 93 ppl 30% 102 ppl 35% 80% Retention (revist in 3days) 23 ppl 25% 25 ppl 25% 80% Revenue Number of ad views per user in a day) 4.2 times 4.2 times 6.1 times 6.1 times 10 times MVPと バージョン2の 比較を行う Compare MVP and its Ver. 2 Copyright 2017 Masayuki Tadokoro All rights reserved
  • 95. Introducing split test/cohort report makes short term work annoying somehow. However, implementing a system that values your work quantitatively will help you shorten the time in the long term. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 96. What you want to learn Increasing data volume customers can add one time will help customers feel less tiresome in using the product See whether customer retention rates are improved by it. Verified story to learn What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved 5 ppl per a day/1 weeks Users add 30MB (3 times of by watching an ad) by answering questionnaires Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in 3 days after registration Revenue: Average # of ad views per a day Write down quantitative results
  • 97. Interview with customers who have actually used version 2. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 98. What you want to learn Increasing data volume customers can add one time will help customers feel less tiresome in using the product See whether customer retention rates are improved by it. Verified story to learn What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved 5 ppl per a day/1 weeks Write down quantitative reusults Users add 30MB (3 times of by watching an ad) by answering questionnaires Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in 3 days after registration Revenue: Average # of ad views per a day
  • 99. What you want to learn Increasing data volume customers can add one time will help customers feel less tiresome in using the product See whether customer retention rates are improved by it. Verified story to learn What KPIs do you measure to verify? What you build Cost & time to build Quantitative result Qualitative result What you learned What you want to learn from the next sprint Iteration Canvas Copyright 2017 Masayuki Tadokoro All rights reserved 5 ppl per a day/1 weeks Write down what you learned Users add 30MB (3 times of by watching an ad) by answering questionnaires Activation: Sign up rates, rates of completion of the sequence Retention: Rates of re-logging in 3 days after registration Revenue: Average # of ad views per a day
  • 100. The batch size of iteration tends to be bigger as time passes As the batch proceeds to next phase, addition, redos and delays occurs. To make the overheads smaller, the butch will get bigger naturally. Copyright 2017 Masayuki Tadokoro All rights reserved Important Points
  • 101. Story Kanban Board Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Story 2-2 Story 2-3 Story To shorten time, renew 2 stories (features) to make them one and examine it Story Story Story Story Story Story (Up to 3)(Up to 3) (Up to 3) (Up to 3) (Up to 3)
  • 102. Story Kanban Board Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Story 2-2 Story 2-3 StoryStory Story Story Story Story (Up to 3)(Up to 3) (Up to 3) (Up to 3) (Up to 3) Make the batch size bigger and verify at one time
  • 103. Backlog In Progress Completed Validation Backlog story Extract stories that you want to learn in the iteration Build Build completed launch Quantitative analysis (split test, cohort analysis) Qualitative analysis/introspection by customer interview MVP story Story 1-1 Story 1-2 Copyright 2017 Masayuki Tadokoro All rights reserved Story 2-2 Story 2-3 StoryStory Story Story Story Story (Up to 3)(Up to 3) (Up to 3) (Up to 3) (Up to 3) Not sure which story (function) is more effective… Story Kanban Board
  • 104. Fall into the death spiral of the big batch. Making a batch size bigger will keep you from achieving the objective of learning something in each iteration. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 105. In case you add a simple feature that has minimal production costs, add it to the issue board. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 106. No Yes Add it on Story Kanban board as story Launch MVP and conduct qualitative and quantitative analysis Achieved PMF? Can improve it if continuing to use the same business model? Conduct transition of the organization to scale Yes Consider to pivot No Yes No Flow chart to decide the next step. Yes No Add it on issue Kanban board as feature Takes amounts of features/issues? Yes No Takes an amounts of features/issues? Verify UX/function improvement of existing features Provide enough values with only existing features? Verify product iteration added features Copyright 2017 Masayuki Tadokoro All rights reserved Add it on issue Kanban board as feature Add it on Story Kanban board as story Add it to the issue Kanban board if it has minimal production costs
  • 107. As a timing of implement is coming up, adopt a just-in-time method where you produce a needed feature when it's needed based on an analysis to verify which items are needed. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 108. ・You can analyze based on the latest and relevant information (enforcing you to answer the reason why you are doing it now) ・Developers can learn more from customers ・It may keep you from reworking ・You may verify qualities of each batch earlier using split test/cohort analysis The effect of the just-in-time method Copyright 2017 Masayuki Tadokoro All rights reserved
  • 109. Product Market-Fit Risk (= Risk not to create needed products) Time PMF risk is rising without customer Feedback Reduce the risk that you could produce unneeded products for which you have consumed a lot of resources/time. PMF risk is limited with customer feedback High Low Copyright 2017 Masayuki Tadokoro All rights reserved
  • 110. How to rhythmically manage the just-in-time method. ・Utilize issue Kanban board ・Conduct daily stand up ・Proceed development based on customer interview ・Develop avoiding documentation *Nevertheless, document customer insights to capitalize/share in company Copyright 2017 Masayuki Tadokoro All rights reserved
  • 111. Backlog In process Completion Validation Backlog story Building Build completed launch Quantitative analysis (split test, cohort analysis) Features Features Features Put features in order of priority Copyright 2017 Masayuki Tadokoro All rights reserved Utilize the Issue Kanban Board
  • 112. Backlog In process Completion Validation Backlog story Building Build completed launch Quantitative analysis (split test, cohort analysis) Features Features Features Copyright 2017 Masayuki Tadokoro All rights reserved Set a limitation similar to the Story Kanban board (Up to 3) (Up to 3) (Up to 3) (Up to 3) Utilize the Issue Kanban Board
  • 113. Backlog In process Completion Validation Backlog story Building Build completed launch Quantitative analysis (split test, cohort analysis) Features Features Features Copyright 2017 Masayuki Tadokoro All rights reserved (Up to 3) (Up to 3) (Up to 3) (Up to 3) Feature Feature Feature Feature Feature Feature In case there is a bottle-neck, use resources and increase throughput Utilize the Issue Kanban Board
  • 114. Share the development process in daily stand up meetings. 3 things to share in a daily stand up: ①What you did yesterday ②What you will do today ③Troubles that may keep your team from working efficiently Copyright 2017 Masayuki Tadokoro All rights reserved
  • 115. Stop documentation in developing. You may try to work in accordance with the document. Copyright 2017 Masayuki Tadokoro All rights reserved
  • 116. Qualitative feedback from customers Team Quantitative data Accumulate it for company learning Learning accumulation Copyright 2017 Masayuki Tadokoro All rights reserved Document customer insights to share in your team.
  • 117. Rotate product iteration with a rhythm. Copyright 2017 Masayuki Tadokoro All rights reserved

Editor's Notes

  1. seedの段階
  2. Use for next iteration by measuring data
  3. たとえば、Gameが大好きだけど、
  4. http://500hats.typepad.com/500blogs/2007/06/internet-market.html
  5. 5000人 500人 50人 25人 5人
  6. Ppl = people
  7. 行動しやすさ、わかりやすさ、チェックしやすさ
  8. You can not set perfect KPIs at beginning You want to continuously verify whether you are measuring appropriate KPI in each stage for your business
  9. Throw vanity metrics away Page view: Number of visitors to the page; it is vanity metrics if you do not set ad-sense or banner ads. Number of visitors: It is vanity metrics since you do not know what types of people are coming how frequently. Unique visitors: It is vanity metrics since you do not know why users came and what they have done It is meaningless if it is just a number. Key point is how many people will take an action when you post something on social media stream Visit duration: You have to measure which duration for specific pages. If customers are staying FAQ or support pages for long time, it indicates that usability is bad Sign-up number: It is meaningless if you can not verify open rate of e-mals
  10. With vanity metrics, you will be reassured, but it won’t provide you clear guidance what you should do next
  11. https://blog.kissmetrics.com/throw-away-vanity-metrics/
  12. Ppl = people
  13. Set up one-on-one interview in order to get feedback. Avoid one-to-N interview, since you cannot hear open opinions P163 start-up owener’s manual
  14. Need to be revised The procedure of the KJ method. 1 Collect interview data. 2 Categorize the data into small units. 3 Organize these units into groups. 4 For each group, add an appropriate label. 5 Put into a logical structure so that you can explain logically. 6 Specify what the true cause of the issue is. インタビューの分析:KJ法を行う ユーザビリティエンジニアリング 1099
  15. http://blogs.wsj.com/accelerators/2014/06/03/jessica-livingston-why-startups-need-to-focus-on-sales-not-marketing/
  16. http://genius.com/Emmett-shear-lecture-16-how-to-run-a-user-interview-annotated
  17. http://blogs.wsj.com/accelerators/2014/06/03/jessica-livingston-why-startups-need-to-focus-on-sales-not-marketing/
  18. Ppl = people
  19. Ppl = people
  20. 新たな知は、経験に基づいて暗黙のうちに持つ主観的な「暗黙知」と、言葉で表現できる客観的な「形式知」が、対話を通して相互に変換し、スパイラルに循環していくなかで生まれる。 http://systemincome.com/tag/野中郁次郎 タグ&個別ページへのリンク
  21. PMFがないと、製品の価値が顧客につたわならい 口コミが広がらない、利用が加速しない。 メディアの評判が最低、 販売サイクルに時間がかかる PMFがあるとすぐにわかる 製品を作ると顧客が買いに来る サービスを追加すると利用が拡大する。 営業や顧客サポートのスタッフが雇える 記者から連絡がある マークアンダリーセン
  22. ー十分に痛みのある課題の存在が存在することを確証した ー課題を解決するのに必要最小限の機能が明確にできた ーソリューションを使ってくれそうな顧客プロフィールが特定できた
  23. ショーンエリスのテスト <プロダクト>が使えなくなったときにどう思いますか? 1 非常に残念 2 少し残念 3 残念でない 4 すでに<>を使っていない 40%以上のユーザーが非常に残念と 答えたのであれば、この”絶対に必要”な製品は 今後も継続的に顧客を獲得できる 40%という数値は数百社のスタートアップを参考にして 決めたものだ。 40%以上を獲得したスタートアップは、 ビジネスを継続的に拡大している 40%を大きく下回ると、たいてい苦戦している アクティベーションのすんだユーザーのうち毎月40% 以上定着すれば、 初期のトラクションがあることになる
  24. seedの段階
  25. seedの段階
  26. Ppl = people
  27. 30MB
  28. Ppl=people
  29. Ppl=people
  30. Ppl=people
  31. Ppl=people