Every product decision is an experiment. If we can reframe our thinking from launching campaigns and building features to challenging assumptions and running experiments, then we can increase our productivity and get better results with less effort. Grace will share a structured process for designing effective experiments, and talk about some of the common pitfalls and challenges teams face when running experiments in their companies. Using examples from her work helping product teams within large companies, Grace will reveal a few ways to overcome these challenges.

1. Designing Effective
Experiments
Grace Ng | @uxceo
Test your business idea in just 5 minutes

2. Photo Q&A

3. Textual search is limiting.
How can I search a visual query?
@uxceo

4. What is this?
@uxceo

5. @uxceo
Minimum Viable Product!
An experiment that uses the least amount of effort
to get the most amount of customer feedback

6. @uxceo
Minimum Viable Product!
An experiment that uses the least amount of effort to test the
Riskiest Assumption against real customer feedback

7. Do people even have visual questions to post?
"
Riskiest Assumption: Have photo Qs to post.
Ask Question
Upload Photo
@uxceo

8. What kind of bird is this? (Is it even a bird??)
@uxceo

9. Will there be people who are able to answer the photo
questions that are posted?
"
"
Riskiest Assumption:
Someone can answer within given time.
@uxceo

10. What kind of bird is this? (Is it even a bird??)
@uxceo

11. Will this scale? Can this be replicated?
@uxceo

12. @uxceo

13. @uxceo

14. @uxceo

15. BUILD
LEARN MEASURE
@uxceo

16. I Don’t
Care...
@uxceo

17. @uxceo

18. $2M. 2 Years.

19. $30M

20. Mistakes

21. Effective Mistakes
Interviews
1. Tested the wrong assumptions
@uxceo

22. Effective Mistakes
Interviews
"
2. Asked the wrong questions
@uxceo

23. Effective Interviews
"
Mistakes
"
3. Didn’t define success
@uxceo

24. Effective Interviews
"
"
"
Mistakes
4. Don’t need to build to test
@uxceo

25. http://bit.ly/xB0D @uxceo

26. @uxceo

27. Case Study
@uxceo

28. Founders UX Help
@uxceo

29. Boot-strapped
startup
founders
@uxceo

30. Boot-strapped
startup
founders
Getting
UX
feedback
Boot-strapped
startup
founders
Getting
UX
feedback
@uxceo

31. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
@uxceo

32. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
@uxceo

33. How How to to MEASURE measure success?
success?
1. Cost
• Project Cost
• Opportunity Cost
"
2. Existing metrics
• Business KPI
• Specific user behavior
@uxceo grace@QuickMVP.com
@uxceo

34. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
@uxceo

35. Exploration Interviews
@uxceo

36. Effective Interviews
1. I’m doing some research around...
2. Have you ever [experienced problem]?
3. Tell me about that time, what happened?
4. What did you do to resolve it?
5. How satisfied were you with the outcome?
6. If unsatisfied, how else have you tried to solve it?
@uxceo

37. Collect Data
• Level of pain/frustration
"
• Frequency of encounter with pain
"
• Existing behavior to solve problem
"
• Time/money spent searching for a solution
@uxceo

38. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo

39. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
@uxceo

40. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo

41. @uxceo grace@QuickMVP.com
$10
@uxceo @uxceo

42. No Backend
@uxceo grace@QuickMVP.com
@uxceo
@uxceo

43. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo

44. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting

45. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10

46. Seriously?
@uxceo

47. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!

48. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!
No!
5 yrs exp.
Established

49. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting

50. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Give quick
online
feedback
Pitch:
60%

51. Pitch
@uxceo @uxceo grace@QuickMVP.com
@uxceo

52. Google Ads
@uxceo @nycuxpa #ngnycuxpa
@uxceo

53. No Backend
@uxceo

54. The 5 Minute Technique
Quick MVP
@uxceo grace@QuickMVP.com
@uxceo @uxceo

55. 3 Data Points
@uxceo @uxceo @nycuxpa #ngnycuxpa
@uxceo

56. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!
No!
5 yrs exp.
Established
Give quick
online
feedback
Pitch:
60%
37/184
emails
Persevere
Started
referring
friends

57. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Give quick
online
feedback
Pitch:
60%

58. Boot-strapped
startup
founders
Getting
UX
feedback
No
reliable
source
Interview:
6/10
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Pay for
virtual UX
feedback
Pitch:
8/10
@uxceo
UX
Designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Will
consult
virtually
Pitch:
8/10
Transition
to UX
designers
Getting
clients for
quick UX
consulting
Virtual
UX
feedback
Give quick
online
feedback
Pitch:
60%
Boot-strapped
startup
founders
Getting
UX
feedback
Virtual
UX
feedback
Online
platform is
sufficient
Concierge:
60%
satisfied

59. Customer Submits Request UX Designer gets email and
submits feedback
Concierge
@uxceo grace@QuickMVP.com
@uxceo

60. Customer Submits Request UX Designer gets email and
Manually
submits feedback
@uxceo grace@QuickMVP.com
@uxceo

61. No
reliable
source
Interview:
6/10
9/12
Persevere
Friends.
Classes.
Pay for
UX help
@uxceo
Pay for
virtual UX
feedback
Pitch:
8/10
10/10
Test
2nd side
of market
Don’t mind
virtual.
It’s cheap.
@uxceo
Will
consult
virtually
Pitch:
8/10
0/10
Pivot!
No!
5 yrs exp.
Established
Give quick
online
feedback
Pitch:
60%
37/184
emails
Persevere
Started
referring
friends
Online
platform is
sufficient
Concierge:
60%
satisfied
20%
Iterate
Wanted
more
function-ality

62. High Frequency
Low Pain High Pain
Low Frequency
Focus
@uxceo

63. Grace Ng | @uxceo
Test your business idea in just 5 minutes