Hypothesis driven design is a powerful method to ensure features you build in your product are valuable and well evidenced. This quick presentation, delivered internally to DWP Digital colleagues, gives tips on writing high quality hypotheses to work from.

1. Writing effective
design hypotheses
Tom Adams, Senior User Researcher, DWP Digital
@sonofswiss

2. Hypotheses
“Hypothesis driven design
helps teams take calculated risks to
move a product forward and explore
different solutions”
Ben Holliday

3. Hypotheses
Hypothesis driven design
• Adds structure and testability to product design activities
• Reduces project waste and increases user value
– ensures we spend time designing and building good
ideas that we are confident will bring user and
business value

4. Hypotheses
Why design with hypotheses
rather than user stories?
• Hypotheses have no size requirement
– They can be small (e.g. equivalent to one user story)
– They can be large features (e.g. equivalent to multiple
user stories or an epic)
• They allow product design to be holistic
– Think of wider user journeys and experiences, rather
than focusing on single features
(Note – hypotheses can drive all design and development, or they can be
used in a design phase with user stories still used for development. There is a
lot of debate and discussion about agile design and process that is out of
scope for this presentation!)

5. Hypotheses
A hypothesis is a way of
structuring ideas to ensure
they are good ideas

6. Hypotheses
What makes an idea a good idea?
• It is clear and sets a context and direction for the team to
design around
• It is based on something we know to be true
• It has a positive outcome for users and the business
• It is testable – it can be proved right or wrong

7. Hypotheses
Example – ‘due dates’
in activity lists

8. Hypotheses
Context
• In the Counter-Fraud and Error
Management Service we’re building,
staff need to know when to perform
particular tasks. We tried using ‘social
media style’ relative dates to tell staff
when particular tasks were due. In
usability testing we saw it wasn’t
successful – so we needed a new
idea.

9. Hypotheses
1.1 The format of due dates is confusing
• The relative dates (11 days ago, 1
day 6 hours from now) are confusing
and cause user error
• Date formats are sometimes
inconsistent (“1 day from now” vs
“1d”), and this causes confusion
“‘2d’ - is this the second
interview?”
Team Leader
“‘7 hours from now’.
What does that mean?”
Team Leader

10. Hypotheses
The idea
“We should show due dates as real dates, in the GDS-
recommended format (5 Feb 2018) instead of relative dates
(15 days from now)”
• This idea is clear – but doesn’t capture why we think it’s a
good idea, what value we think it will bring if we do it, and
how we know whether it will have worked or not.

11. Hypotheses
What makes an idea a good idea?
• Let’s do [idea]!
• If we do [idea], we will get [outcome].
• Because [something we know], if we do [idea], we will get
[outcome].
• Because [something we know], if we do [idea], we will get
[outcome], and this will be proven when we see
[evidence].

12. Hypotheses
Example
We’ve seen lots of users in usability testing struggling to
understand when their activities are due when they see the
relative date format (“15 days from now”). Also, we know that
actual dates were well understood in the prototype, and we know
that GDS-formatted dates (“5 Feb 2018”) have tested well across
government. So if we show due dates as real dates in the GDS
format, users will understand better when their activities are due
and make less mistakes. This will be proven when we see no
misunderstanding and mistakes around due dates in usability
testing.
• This contains all the content we need to assess whether the
idea is good – but it is not clear and easily understood, as it
isn’t structured.

13. Hypotheses
Because [things we know or have seen]
We believe that [idea]
Will result in [outcome]
We’ll know this is true when [evidence]
(Note – there are lots of different hypothesis formats out there. This is my
preferred structure, but as long as you cover the important content, search
Google and try a few to find which you’re comfortable with!)

14. Hypotheses
Because [things we know or have seen]
We believe that [idea]
Will result in [outcome]
For [users]
We’ll know this is true when [evidence]

15. Hypotheses
Because:
• we’ve observed the majority of users over multiple studies struggle to correctly
state when activities are due,
• they have said they don’t understand the current style of ‘relative’ date formats,
• users did understand the absolute date formats used in the prototypes,
• we know that the GDS pattern of date formats are well tested and well trusted;
We believe that:
• changing the format of all dates across the system to something more like the
GDS pattern;
Will result in
• users intuitively understanding when their activities are due, so speeding them up
and reducing chance of error;
For:
• agents, team leaders and future users;
We’ll know this is true:
• when the majority of users in usability testing correctly state when their activities
are due.

16. Hypotheses
How do we write strong
content for each line of the
hypothesis?

17. Hypotheses
Because [things we know or have seen]
We believe that [idea]
Will result in [outcome]
For [users]
We’ll know this is true when [evidence]

18. Hypotheses
“Because”
Strong (eg)
• User needs
• User observations
• Data
• Business needs
• Technology needs
Weak (eg)
• Something that
happened on
another
project/different
users
• Business wants
• User wants
Weaker (eg)
• Hunches
• HiPPOs
• ‘Best practise’
This is “Things we know” – evidence from data,
observations of users, etc.

19. Hypotheses
“Because”
• The more ‘things we know’, the stronger the idea
– E.g. If we observe users having a problem, and we
know there is a technology issue, and we know there
is a strong business need around this feature – the
product owner will likely prioritise it above an idea that
has had no major observed user problem.
• Hunches/un-evidenced ideas should lead to further
research, not go straight to design

20. Hypotheses
Because [things we know or have seen]
We believe that [idea]
Will result in [outcome]
For [users]
We’ll know this is true when [evidence]

21. Hypotheses
“We believe that”
This is the idea.
• It should be directional and clear
• But (unless the solution is so clear and obvious that a
design process would be a waste of time) it shouldn’t
generally dictate or pre-suppose an entire solution
– This is the starting point for an iterative design
process

22. Hypotheses
Because [things we know or have seen]
We believe that [idea]
Will result in [outcome]
For [users]
We’ll know this is true when [evidence]

23. Hypotheses
“Will result in”
This is the expected outcome.
• It should state some kind of value to users or the
business
• If there is no value in doing this idea, why would we
waste time working on it?

24. Hypotheses
Because [things we know or have seen]
We believe that [idea]
Will result in [outcome]
For [users]
We’ll know this is true when [evidence]

25. Hypotheses
“We’ll know this is true when”
This is the test.
• What evidence will show that the idea has worked/hasn’t
worked?
• It should be clear and (wherever possible) not open to
interpretation
– i.e. when we test the idea, it should be clear and non-
debateable that the idea has succeeded or failed
– Best way to do this is releasing something and testing
with real data (e.g. A/B testing)
– Usability testing an idea is always more open to
debate, and successful ideas should be re-tested with
data when released to live

26. Thanks
@sonofswiss