The presentation from my talk at #LeanDayWest, September 17, 2013 in Portland, OR. Research Rebooted: Market Research is Broken, How Lean Can Help Fix It.

1. Research Rebooted
Why Most Market Research Is Broken,
And How Lean Can Help Fix It

2. what’s wrong with
research?

3. the value is broken.

4. $30,000,000,000+

5. time.
it’s in a box.
and it slips.

6. information request
research RFP
study design
recruitment
fielding
Analysis
Brand/product team
research & insights
research company
recruiter/panel
moderators
research company

7. The CEO’s Kids
“I keep a lot of kids around me and ask them questions and peek around
the corner when they think I'm not listening. My master plan was I had a
lot of kids. I have six kids. Three girls that are six, a 15-year-old, 18-year-
old and 22-year-old. I have the perfect research team in my house."
http://acupofteawithphd.wordpress.com/2013/06/19/live-from-cannes-cannes-lions-2013-so-farpart-4-its-diddy/

8. the design is broken.

9. The fear of small numbers
Managers worry about the
signal-to-noise ratio.
The smaller the sample, the
more we worry about noise
(wrong people, inaccuracies or
untruths, groupthink, etc.)
So they compensate with bigger
sample sizes.
But more numbers
make more noise.

10. over targeting leads us astray
“[M]ost companies with a practiced discipline
of listening to their best customers and
identifying new products that promise greater
profitability and growth are rarely able to
build a case for investing in disruptive
technologies until it is too late.”
- Clay Christensen, The Innovator’s Dilemma

11. research is often designed to confirm biases.
when it doesn’t, it’s painful.

12. our customer or our design target?

13. research ignores switchers,
rejectors, fringe & new users
“Generally, disruptive technologies
underperform established products in
mainstream markets. But they have other
features that a few fringe (and generally new)
customers value.”
- Clay Christensen, The Innovator’s Dilemma

14. the process is broken.

15. garbage in, garbage out
“The research manager’s job is
to manage the hell out of the research.”
- Michelle Crawford

16. quant über alles

17. packing 10 lbs into a 5 lb bag
30 minute surveys
2 hour focus groups
10 page discussion guides
Too many questions, too much data,
not enough understanding

18. stockpiling data

19. trapping insight in
powerpoint decks

20. It's really hard to design products
by focus groups. A lot of times,
people don't know what they want
until you show it to them.
- Steve Jobs

22. you ≠ steve jobs.

23. research ≠ design
Research is the process by which we
understand problems.
Design is the process by which we
solve problems.

24. how lean can help fix
research.

25. what is a lean company for?
1. To specify customer value.
2. To identify & implement the actions that
create that value.
3. To remove anything that doesn’t create value.
4. To analyze the results (and repeat).

26. why does feedback &
customer pull matter?
Feedback and testing (a.k.a., research)
help teams settle questions and make decisions
based on evidence not preferences.
Continuous feedback helps teams see progress.

27. Go and see.
This is the new intimacy.

28. how do we do lean
research?

29. the owner has to change

30. empower the
research manager
We have to transform the research manager
from a project manager into a Scrum master.
Their role is to remove obstacles to progress,
and to facilitate communication.
Their responsibility to the team is to facilitate
hypothesis testing, and to create
opportunities to challenge assumptions.

31. validation

32. the tools have to change

33. use right-sized tools

34. Use the tools
closest to hand

36. Talk to people where
it’s relevant or convenient
i.e., NOT HERE

37. talk to them here.

38. the whole team can do this.
http://giffconstable.com/2010/07/12-tips-for-early-customer-development-interviews/

39. the flow has to change

40. Do less, more often
Lean research is about continuous
customer, partner & team feedback.

41. set people up for success

42. the output has to change.

43. data doesn’t “tell” you
what to do.

45. nothing beats a conversation.
To make research
meaningful to the
whole team, the
whole team needs:
Investment
Access
Accountability

46. forget the horse’s mouth
Don’t count on users to make your
decisions for you - that’s not their job.
Don’t let the report be the end of your
learning process - it’s just a way to
memorialize what you’ve learned and provide
a reference when you encounter similar
issues again.

47. information should be
reusable, not disposable.

48. watch this space.
and thanks, this was fun.