1. Handicapping & Dark Horses
What Design Thinking Can Learn from Horse

6. 2 Techniques

7. #1 Handicapping

8. handicapping:
the art of evaluating a horse’s
performance in order to determine
the outcome of a race

11. handicap the ideas

13. running a rapid experiment

14. rapid experimentation:
using prototypes to evoke
behaviors that quickly tell us
whether our ideas meet users’
needs

15. rapid experiments
usability test
survey
feedback methods
what will someone
actually do
what they might do

16. rapid experiments separate
what customers say from
what they do in the real world

17. what is an experiment?
• a scientific procedure—a controlled empirical test of a hypothesis
• hypotheses include (never null!):
-A causes B
-A is better, bigger, faster than A’
-A changes B more when we do/provide X
• requirements:
– independent variable that can be manipulated
– dependent variable that can be measured
– random assignment to condition (conservatively)

20. +

22. What can design experiments test?
• are people interested?
• will they do/use this given all other
choices/demands available?
• does it meet the need you designed for?
• does it have the desired (and not undesired)
effects?
• can people figure out how to use/do it?
• which design is better (at any of the above)?

23. How do you do this at home?
1. Pick an idea
2. Make a list of all the questions you have about
the efficacy of the idea
3. Select the most critical question to success
4. Generate a hypothesis
5. Design an experiment to test your hypothesis
6. Create the prototype to support the experiment

24. Before you move into prototyping, make a plan:
• What questions do you have about the
effectiveness of the ideas?
• What is my hypothesis about what will happen?
• What kinds of observations would validate my
hypothesis?
• What prototype(s) do I need to create for these
observations?
• How should I run this experiment?
Use the prototype to
• Create an experience that evokes the desired behaviors
• Evaluate your working hypotheses
Don’t create the prototype before designing your experiment

26. Make a list of questions
Concepts we generate often have implicit
assumptions about how people will
respond and what people will do
What are the crucial questions that could
make or break the success of your idea?

27. Select the most critical question
Will people be more likely to select a
rental if it has a nice photo?
Will homeowners be willing to let
photographers photograph their space?
How expensive is it create this service?
Will the photos really look better?

28. Generate a hypothesis
A hypothesis is your prediction about how
people will behave.
A lot more people will rent an apartment
that has been professionally
photographed than one that has not.

29. Design an experiment
• the goal is to evoke “real” behaviors in
“real” situations
• the scenario must be
– believable
– immersive
– natural
• and, allow you to test/measure what
you need to
• a word on “confederates”

31. Plan for measurement
people are notoriously bad at
predicting (and also
remembering/reporting) their
own behaviors

32. Plan for analysis
• before conducting your experiment, figure out how
you’re going to analyze your data
• what conclusions will you be able to draw from the data?
normal photo pro photo
4 5
Did they
rent the
apartment?

33. Design your experiment
– A/B Test
– Show ½ the people the a rental with normal
photos, ½ the same rental with professional
photos.
– Constrain to one geographic area
– Select 2 popular and 2 unpopular apartments.
– Measurement: What percentage of people who
view the apartment rent it in condition A vs. B.

34. Create the prototype

35. run the experiment
&
analyze your results

36. Tip: You can follow up with
qualitative interviewing

38. #1 Handicapping

39. Dark Horse
#2 Dark Horses

40. dark horse:
a contestant that seems unlikely
to succeed, but emerges to
prominence against all odds

41. dark horse idea
1. dark: explores a space that is risky, difficult,
radical, or orthogonal
2. brainstormed after more traditional ideas
3. refined enough that can be prototyped and
tested with a rapid experiment (can’t be
infeasible)

42. rational
1. Designers need to preserve ambiguity to leave
themselves open to new ideas
2. Taking the dark horse idea to the point you can
evaluate it keeps the design space from shrinking
too fast
3. Even if the dark horse doesn’t work, you will gain
insights that will make your final design better

43. Example 1
Design the next generation
cinema/broadcasting
imaging system

44. Original idea: Immersive movie watching
experience where viewer’s head and eye position
affect the perspective and focus of a scene
Dark Horse: What if the camera was
everywhere?
Rigs of multiple Kinects and software capture
3D info of a scene for post capture scene
manipulation.

45. Example 2
Design a system to
improve street navigation for
people on vacation

46. Original idea: Hotter/Colder
Phone vibrates differently as you are
getting closer or farther from the
direction you should be going.
Dark Horse: Stalker Navigation
No maps, crowdsourced directions,
follow the stranger.

47. Dark Horse
#2 Dark Horses

48. Handicapping & Dark Horses
What Design Thinking Can Learn from Horse