Hypothesis Driven Design:
Get Experimental
FITC Spotlight UI/UX 2018
Leslie Predy
About me
About me
In this talk
• Background
• Why you should use hypothesis driven design
• When a hypothesis is useful
• How to define and test a hypothesis
• Takeaways
The problem with many startups is that you spend months or years doing
research, writing requirements, designing and building software…
…and discover no customer or user cares.
—Will Evans
Traditional Product Development
Will Evans
Example: Microsoft Bob
Lean UX
Jeff Gothelf
Scientific Method
It’s not scary!
Design Hypothesis
Take an assumption that your concept or design is based on,
something that you believe to be true. Phrase it in a way that allows it
to be proven or disproven using research or experiments.
The results of these experiments will tell you whether understand your
user’s behavior and the potential or the pitfalls of your concept.
What is a design hypothesis?
Scientific Method
Collaborative
• Shared understanding of goals
• Get cross functional team buy in
Evidence based
• Informed insights back up your design decisions and roadmap
Holistic
• Give structure to design explorations
• Quick validation of concepts
Why use design hypotheses?
When to use design hypotheses
UK Design Council
Double Diamond
Is the concept right? Is the solution right?
How To Create Your Hypothesis
Identifying Assumptions
Assumptions
Have a post it party
• Engage with your team
• Identify the 4Ws
• Who, what, where, why
We believe that [creating this experience] for [this persona]
Will achieve [desired outcome].
We will know our hypothesis is valid if [measurable metric].
Writing a hypothesis
Furthermore Design
We believe that having lists of verified hosts for potential renters will
make skeptical renters more likely to book.
We will know our hypothesis is valid:
• If potential renters choose verified hosts over (equivalent) other
hosts for 60% of their bookings.
• If over 50% of undecided renters say they are willing to book with a
verified host.
Vacation Rental App
Example
We believe that having lists of verified hosts for renters will make
renters feel more comfortable about their stay.
We will know our hypothesis is valid if over 80% of renters say their
stay was a positive experience and accurate with the listing.
Vacation Rental App
Example
Test Your Hypothesis
What type data do you need?
Qualitative: Interviews
• Face to face interviews
• Journey mapping
• Co-creation
Qualitative: Observation
Prototypes
Low Fidelity
• Paper prototype
• Test concepts quickly
• No coding
Medium Fidelity
• Invision, Axure, Principle, etc.
• Test interactions
• Validate workflows
Quantitative
Analytics
• Surveys
• Usage data & metrics
• Mixpanel, Fullstory, etc.
We believe that having lists of verified hosts for potential renters will
make skeptical renters more likely to book.
We will know our hypothesis is valid:
• If potential renters choose verified hosts over (equivalent) other
hosts for 60% of their bookings.
• If over 50% of undecided renters say they are willing to book with a
verified host.
Vacation Rental App
Example
We believe that having lists of verified hosts for renters will make
renters feel more comfortable about their stay.
We will know our hypothesis is valid if over 80% of renters say their
stay was a positive experience and accurate with the listing.
Vacation Rental App
Example
Autodesk Fusion Production
Autodesk Fusion Production
Return on research
Adapted from Jakob Nielsen
Return on research
No research = no insights
Learn & Improve
• Your metric may be wrong
• Refine your experiment
• Your hypothesis may be false
• Be prepared to change direction
• Failure is an important part of learning
• Value the gap: ask more questions
Analyze & Refine
There are known knowns. These are things we know that we know. There are
known unknowns. That is to say, there are things that we know we don't know.
But there are also unknown unknowns. There are things we don't know we
don't know.
—Donald Rumsfeld
Be mindful:
• You will never know with certainty how your product will be used
unless you build it
• It’s hard to pick up on weak signals
• The longer you spend on a design, the less likely you’ll be to discard it
• Keep it lean—test often and make it easy so you can experiment
• Don’t stifle creativity, but verify your decisions
Takeaways
Questions?
twitter.com/lpredster
linkedin.com/in/lesliepredy
Thank You

Hypothesis Driven Design: Get Experimental

  • 1.
    Hypothesis Driven Design: GetExperimental FITC Spotlight UI/UX 2018 Leslie Predy
  • 2.
  • 3.
  • 4.
    In this talk •Background • Why you should use hypothesis driven design • When a hypothesis is useful • How to define and test a hypothesis • Takeaways
  • 5.
    The problem withmany startups is that you spend months or years doing research, writing requirements, designing and building software… …and discover no customer or user cares. —Will Evans
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
    Design Hypothesis Take anassumption that your concept or design is based on, something that you believe to be true. Phrase it in a way that allows it to be proven or disproven using research or experiments. The results of these experiments will tell you whether understand your user’s behavior and the potential or the pitfalls of your concept. What is a design hypothesis?
  • 12.
  • 13.
    Collaborative • Shared understandingof goals • Get cross functional team buy in Evidence based • Informed insights back up your design decisions and roadmap Holistic • Give structure to design explorations • Quick validation of concepts Why use design hypotheses?
  • 14.
    When to usedesign hypotheses UK Design Council
  • 15.
    Double Diamond Is theconcept right? Is the solution right?
  • 16.
    How To CreateYour Hypothesis
  • 17.
  • 18.
    Assumptions Have a postit party • Engage with your team • Identify the 4Ws • Who, what, where, why
  • 19.
    We believe that[creating this experience] for [this persona] Will achieve [desired outcome]. We will know our hypothesis is valid if [measurable metric]. Writing a hypothesis Furthermore Design
  • 20.
    We believe thathaving lists of verified hosts for potential renters will make skeptical renters more likely to book. We will know our hypothesis is valid: • If potential renters choose verified hosts over (equivalent) other hosts for 60% of their bookings. • If over 50% of undecided renters say they are willing to book with a verified host. Vacation Rental App Example
  • 21.
    We believe thathaving lists of verified hosts for renters will make renters feel more comfortable about their stay. We will know our hypothesis is valid if over 80% of renters say their stay was a positive experience and accurate with the listing. Vacation Rental App Example
  • 22.
  • 23.
    What type datado you need?
  • 24.
    Qualitative: Interviews • Faceto face interviews • Journey mapping • Co-creation
  • 25.
    Qualitative: Observation Prototypes Low Fidelity •Paper prototype • Test concepts quickly • No coding Medium Fidelity • Invision, Axure, Principle, etc. • Test interactions • Validate workflows
  • 26.
    Quantitative Analytics • Surveys • Usagedata & metrics • Mixpanel, Fullstory, etc.
  • 27.
    We believe thathaving lists of verified hosts for potential renters will make skeptical renters more likely to book. We will know our hypothesis is valid: • If potential renters choose verified hosts over (equivalent) other hosts for 60% of their bookings. • If over 50% of undecided renters say they are willing to book with a verified host. Vacation Rental App Example
  • 28.
    We believe thathaving lists of verified hosts for renters will make renters feel more comfortable about their stay. We will know our hypothesis is valid if over 80% of renters say their stay was a positive experience and accurate with the listing. Vacation Rental App Example
  • 29.
  • 30.
  • 31.
    Return on research Adaptedfrom Jakob Nielsen
  • 32.
    Return on research Noresearch = no insights
  • 33.
    Learn & Improve •Your metric may be wrong • Refine your experiment • Your hypothesis may be false • Be prepared to change direction • Failure is an important part of learning • Value the gap: ask more questions Analyze & Refine
  • 34.
    There are knownknowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know. —Donald Rumsfeld
  • 35.
    Be mindful: • Youwill never know with certainty how your product will be used unless you build it • It’s hard to pick up on weak signals • The longer you spend on a design, the less likely you’ll be to discard it • Keep it lean—test often and make it easy so you can experiment • Don’t stifle creativity, but verify your decisions Takeaways
  • 36.