Data-Driven
Product Management
Practical Ideas and Tools PMs Can
And Should Use to Make Decisions

1
About Me





Who here lives in Arlington? (Vote Dunn!)
MIT mechanical engineer (but I never used it)
7 startups in 15...
3
Most PMs Aren’t Visionaries
 Ideas come from customers, colleagues, and prospects
 Steve Jobs isn’t walking into this pr...
Ideas Are Not the Scarce Resource
 Ideas come in sizes: markets, features, bug fixes, and
optimizations
 They have diffe...
Optimize for Enterprise Value
 The PM’s job is to prioritize
 What’s the North Star for your company?
 Stars are direct...
“That is a knowable fact.”
 What the advocate says







“No one uses that feature”
“Everyone wants this!”
“That b...
Know Which Facts Are Knowable
 Carefully separate opinion from fact, known from
unknown
 Huge, immediate reduction in co...
Know Your Data. Wallow In It.

9
Your Application Database Knows
 Your customers using your app are telling you how they
use it.
 You need to get the dat...
Measure It From The Start
 Your application database can’t tell you everything
 Make an early change that adds data and ...
Measure the Good and the Bad
 You have to know what the problems are
 You have to know when they get worse
Make a Dashboard of It
When Do You Have to Decide?
 Most of the time, the answer is “later”
 Don’t decide until you have to
 This is where the...
Time to Invest!

15
Keep Investing!

16
How Do You Decide?
 Most decisions aren’t reduced to a time series
 Comparing apples, oranges, and bacon
 Your company ...
Find a way to order the data






Whiteboards and stickies
What themes can you find
What time ordering can you find
...
Build a framework
 Whiteboards and stickies – and Excel
 Just make one up










10 points for data loss
1 po...
Customer Pain (in thousands)

<date> – Confidential

20
My Tools
 SQL
 You need access to the data, not reports
 NoSQL has query tools, too

 Text editor
 UltraEdit. Python,...
<date> – Confidential

23
Data Driven Product Management - ProductTank Boston Feb '14
Upcoming SlideShare
Loading in …5
×

Data Driven Product Management - ProductTank Boston Feb '14

1,198 views

Published on

Practical Ideas and Tools PMs Can And Should Use to Make Decisions

Talk given at Boston ProductTank Meetup. http://www.meetup.com/ProductTank-Boston/events/165579612/

Published in: Technology

Data Driven Product Management - ProductTank Boston Feb '14

  1. 1. Data-Driven Product Management Practical Ideas and Tools PMs Can And Should Use to Make Decisions 1
  2. 2. About Me     Who here lives in Arlington? (Vote Dunn!) MIT mechanical engineer (but I never used it) 7 startups in 15 years Career path from support to implementation to QA to PM <date> – Confidential 2
  3. 3. 3
  4. 4. Most PMs Aren’t Visionaries  Ideas come from customers, colleagues, and prospects  Steve Jobs isn’t walking into this product meeting  PMs probe, interpret, and synthesize 4
  5. 5. Ideas Are Not the Scarce Resource  Ideas come in sizes: markets, features, bug fixes, and optimizations  They have different motivations  Increased sales  Higher retention  Lower cost of goods  Unlimited resources, you could do it all – but we don’t have that  Someone has to decide what is next  This is why PMs get paid the big bucks 5
  6. 6. Optimize for Enterprise Value  The PM’s job is to prioritize  What’s the North Star for your company?  Stars are directional – you can’t make a map to get to star  How do you know if you are pointed in the right direct?  How do you know if you are making progress?  How do you compare apples to oranges?  And compare that to bacon? 6
  7. 7. “That is a knowable fact.”  What the advocate says       “No one uses that feature” “Everyone wants this!” “That breaks all the time” “You're not fixing enough bugs” “This problem happens to everyone!” “I’ve heard this request a million times”  What the data says  15% of users click that every week  We’ve had 3 customers ask for this feature  5% of support calls are associated with a bug 7
  8. 8. Know Which Facts Are Knowable  Carefully separate opinion from fact, known from unknown  Huge, immediate reduction in complexity of the decision  Develop a third and fourth category 1. 2. 3. 4. We really don’t know Knowable fact We can know if we do . . . Before we decide, we really should know  A good PM uses all 4 categories to make a decision  This talk is more about 3 and 4 8
  9. 9. Know Your Data. Wallow In It. 9
  10. 10. Your Application Database Knows  Your customers using your app are telling you how they use it.  You need to get the data reproducibly  You need data, not reports  Know what you need to change  Know if your changes actually worked or not 10
  11. 11. Measure It From The Start  Your application database can’t tell you everything  Make an early change that adds data and measurement  Pipeline speed  Funnel shape  Daily activity
  12. 12. Measure the Good and the Bad  You have to know what the problems are  You have to know when they get worse
  13. 13. Make a Dashboard of It
  14. 14. When Do You Have to Decide?  Most of the time, the answer is “later”  Don’t decide until you have to  This is where the art meets the science  Know your downsides and worst-case scenarios, and mitigate them  Watch, and monitor  Agile (“agile”) really shines here  You will have the development bandwidth when you need it  Unfortunately frustrating for many customers and colleagues 14
  15. 15. Time to Invest! 15
  16. 16. Keep Investing! 16
  17. 17. How Do You Decide?  Most decisions aren’t reduced to a time series  Comparing apples, oranges, and bacon  Your company needs all three  Collect all the data you can  Read what the customer said (or potential customer). Talk to them directly.  Talk to the people who interacted with them (support, consultant, sales rep, account manager)  Look at the usage  Look at the market and the competition 17
  18. 18. Find a way to order the data      Whiteboards and stickies What themes can you find What time ordering can you find What pre-requisites can you find Which ideas are both cheap and enable discovery 18
  19. 19. Build a framework  Whiteboards and stickies – and Excel  Just make one up         10 points for data loss 1 point for annoying 1 point per customer affected 3 points per big customer You are the most qualified person to do it See what maps to your intuition, what doesn’t Know the limitations of what you built Iterate 19
  20. 20. Customer Pain (in thousands) <date> – Confidential 20
  21. 21. My Tools  SQL  You need access to the data, not reports  NoSQL has query tools, too  Text editor  UltraEdit. Python, Perl work too  Turn dross into data  Excel  You do know how to make a pivot table, right? Find the lumpy parts.  Can you do vlookups in your sleep? Integrate your data sources  Tableau  Whiteboards and stickies  TheBrain mind-mapping software 21
  22. 22. <date> – Confidential 23

×