This was an Introduction to decision making processes for people who work in Product Management by the Senior Product Strategist at Philosophie, Chris Butler. This talk covered how to align qualitative research with quantitative analytics for better product decisions.

1. Aligning Quantitative Analytics for Better
Product Decisions
/Productschool @ProductSchool /ProductmanagementNY

2. Aligning Quantitative Analytics
for Better Product Decisions
Chris Butler
Links and references available here:
http://goo.gl/c0AfbN

3. Chris Butler
Senior Product Strategist @ Philosophie in NYC
Over 17 years of product and BD
Microsoft, Waze, Horizon Ventures, KAYAK
Started my own company, failed
chrisbutler@philosophie.is
@chrizbot

4. I have opinions about analytics...

5. ...and about research

6. What we will cover
Quantitative crash course
Qualitative crash course
Helpful heuristics for quantitative and qualitative methods
How do you decide what to measure?
How does this aid decision making?

7. Key takeaways
Why and how to collect great quantitative data - discussion of
common problems
How qualitative research and quantitative analytics create a
virtuous cycle of learning
How to apply this towards great product decisions
This is just a jumping off point...

8. Quantitative
Numbers, categories,
ranking, units
Collected automatically
Tells you what someone
did
Doesn’t tell you why
someone does
something
Narratives, observations
Lots of context
Takes more
time/resources to
setup and run
Qualitative
VS

9. Quantitative analytics methods

10. “Good” metrics
Not so confusing that you have to explain it
Comparative (e.g. new vs. churned users)
A ratio or rate (e.g. % monthly active users)
Helps you take action or generates change, not just for
vanity

11. Snapshot

12. Time series

13. The funnel

14. The funnel

15. Behavioral flows

16. Cohort analysis

17. KPIs

18. KPIs
Should be a “good” metric
Should be meaningful to the success of the business
Should be 5 KPIs at most - ideally less
Can change over time

19. Why is building dashboards cowardly?

20. How to get started with implementation...

21. Qualitative research methods

22. “Good” research
Observe people, in their own environments
Don’t ask them leading, biased, or yes/no questions
Do ask them open ended questions, what they have done in
the past
Synthesize in chunks to see patterns

23. Usability and prototype testing

24. Contextual inquiry

25. JTBD interviewing

26. JTBD interviewing

27. Sales safari

28. How to get started...

29. Helpful heuristics

30. Find the minimums and maximums

31. Find the edges

32. Don’t collect everything

33. Switch to qualitative with smaller samples

34. How do you decide what to measure?

35. Qualitative
research
Quantitative
analytics
Product
context

36. Strategy?

37. Simply: why?

38. Work from the high level outcome to the
lower level, “good” metrics...

39. “How laddering” down
Challenge: salespeople don’t have modern tools
Secret (or O): provide tools to automatically collect sales info
Outcome (or KR): increase efficiency for salespeople
Efficiency metric = less time per sale amount
Metrics: sale start, sale end, sale amount

40. How does this aid decision making?

41. Hypothesis tracking
Track major hypotheses on what you build/prototype
What do we think will happen?
Avoid determining next steps (yet)
Validate/invalidate hypotheses
Decide what this changes about your product context

42. Hypothesis framework

43. How do you adapt?

44. Retro the value of your analytics/research
What is measuring important information?
What is not measuring important information?
What should we be measuring?

45. In closing...
Both quantitative analytics and qualitative research are needed to
build great products
Ask good questions to get helpful answers
Constantly learn from your customers
Links and references available here:
http://goo.gl/c0AfbN

46. Thank you!
Links and references available here:
http://goo.gl/c0AfbN

47. Q&A
Links and references available here:
http://goo.gl/c0AfbN