A presentation given at UXNZ 13, on integrating user experience and participatory approaches with traditional evidence-based approaches to design mental health interventions for young people. Presentation given in collaboration with @kittyrahilly and @mariesanicholas from the Inspire Foundation in Sydney, Australia. See full abstract & audio of the presentation http://uxnewzealand.co.nz/uxnz-2013/integrating-ux-evidence-based-approaches/ For more info see More info http://www.smallfire.co.nz/2014/01/25/integrating-user-experience-and-evidence-based-approaches-to-design/

1. Integrating UX
and evidence-based
approaches to design
effective online youth mental
health services with young
people.
@pennyhagen
@kittyrahilly
@mariesanicholas
With much thanks to:
Brain & Mind Research Institute
Young and Well CRC
Victoria Blake, Inspire
Stephen McKernon
And our co-designers

2. What I am talking about today?
User Experience Design Approach
Evidence-based Design Approach
Youth
Mental
Health

3. In collaboration with:
@mariesanicholas
Senior Manager, UX
Research and Service Design
@kittyrahilly
Manager, UX Research
& Service Design

4. The Inspire
Foundation

5. Inspire Foundation

6. What we’ll cover

7. Two approaches to design
User Experience Design Approach
Evidence-based Design Approach

8. User experience design
The design of a product or service that accounts for the needs, expectations, experiences
and perspective of those who will use it, through collaboration with those users.
IDENTIFY &
DEFINE
CONCEPT
BUILD
LAUNCH &
EVALUATE
Understand domain
from a user
perspective.
Identify area of
greatest
opportunity.
Develop and test a
range of concepts
with users.
Identify strongest
proposition.
Build, test, iterate
and evaluate with
users against user
goals.
Launch.
Potentially track
usage and look to
optimise.

9. Evidence-based design.
The design of an intervention on the basis of research data, past evaluations, health
planning frameworks and theories of behaviour change.
IDENTIFY
PROBLEM
DEFINE FOCUS
AND APPROACH
BUILD & TEST
EVALUATE &
PUBLISH
Epidemiological
analysis
Review literature
Build program logic
Monitor outcomes
Assess and select
theory of change
Evaluation
framework tools
Publish results
Identify outcomes
and impacts
Build experiments
Consult experts
Review existing
national policies
and strategies
Review for safety
and ethics

10. Why integrate?
To achieve greater impact through interventions that
are based on sound evidence and are also engaging for
young people .

11. Why integrate?
Re-thinking clinical models and
service delivery within an online,
anonymous, self-service context.
Ways to actively involve the young
people who will use the service in
the design of the service.
A specific focus on the
perspectives and experiences of
the end-users of a service.
Connecting the models and
theories of evidence-based
approaches with the lived
experiences of the young people.

12. How do we do it?

13. 1. Translate the model
Understanding how, when, where and
in what way mental health issues can
be broached with young people.

14. 1. Translate the model
From mental health to mental fitness,
handling pressure, building
confidence and taking control.

15. 2. Re-interpret the model
Mapping sleep factors to the evidencebased model

16. 3. Identifying fail points
Understanding where a negative
experience will likely cause the
intervention to fail
Image credit: http://www.flickr.com/photos/frak_tal/2455855855/

17. 4. Redirect opportunities
Understanding what is a priority to
young people

18. 5. Work with the evidence

19. How we do it
#1 Translate models from the evidence-base
#2 Re-interpret models from the evidence-base
#3 Identify fail points
#4 Re-direct opportunities
#5 Work with the evidence

20. Challenges

21. Challenge 1:
Different ways of measuring
success means different priorities.

22. Challenge 1:
Different ways of measuring
success means different priorities.
Evidence-based
success =
User Experience
success =
The model applied in the way
intended with a positive result that
can be correlated to the
intervention.
An intervention that is used and
useful to those who it was
designed with and for.

23. Challenge 1:
Different ways of measuring
success means different priorities.
Evidence-based
success =
User Experience
success =
Model must not be altered in order
to be measured.
Model needs to be flexible in
order to be usable.

24. Challenge 2:
Our forms of evidence and
how we manage them are not
the same

25. Challenge 2:
Our forms of evidence and
how we manage them are not
the same
Part 1. Our methods tend to be qualitative and largely
visual
Part 2. In comparison our methodology is not as
rigorous (partly because it hasn’t had to be)

26. Evolving our tools and
methods

27. 01 design tools to
Adapting
connect and integrate both
perspectives as inputs for
design.

28. Personas and scenarios include
mental health status, and
demonstrates how behaviours and
the theory of change play out in the
real world.

29. Personas and scenarios include
mental health status, and
demonstrates how behaviours and
the theory of change play out in the
real world.

30. User Journey maps help show how
theory of change will be supported
through the intervention and how this
connects to the intended user
experience.
Reproduced from: Hagen, P, Collin, P, Metcalf, A, Nicholas, M, Rahilly, K, & Swainston, N 2012, Participatory Design of evidence-based online youth
mental health promotion, prevention, early intervention and treatment, Young and Well Cooperative Research Centre, Melbourne.

31. 02
Evolving how method and
findings are communicated

32. There are cycles of input and
validation with young people
throughout the design process.
Images from: Hagen, P, Collin, P, Metcalf, A, Nicholas, M, Rahilly, K, & Swainston, N 2012, Participatory Design of evidence-based online youth mental
health promotion, prevention, early intervention and treatment, Young and Well Cooperative Research Centre, Melbourne.

33. Showing the source of all insights and
presenting individual project findings
in the context of all other research and
findings from other projects.
Insight
ReachOut guiding principles
The guiding principles describe the underlying factors that
steer the service, irrespective of specific project goals,
constraints and resources.
All aspects of ReachOut should meet these principles, or at the
very least, never contradict them.
• Focus on the benefit to young people and their friends
• Be relevant to them and their life
• Reach out to where young people are
Sources
Researchers: Pollinate Market Research, Mariesa Nicholas,
Nathalie Swainston, Kitty Rahilly
Project/Program: ReachOut Next Gen
Method:
Market Research Focus Groups (see Project Happy: Group
Testing Method & Questions)
Thematic Analysis of Transcripts
Sample (n = 29):
18 Males, 11 Females
Existing and non-ReachOut.com users
Diversity across the Satisfaction With Life Scale (Diener et al
1985)
Divided into 3 groups:
Existing ReachOut.com Users (n = 6)
SWLS Slightly Dissatisfied (n = 7, SWLS)
SWLS Moderately Dissatisfied (n = 5)
Tested:
Focus group method with young people aged 16 to 25 mixture
male and female.
ReachOut Next Gen Workshop 1 (n=21)
ReachOut Next Gen Workshop 2 (n=19)
ReachOut Next Gen Youth Ambassador Workshop (n=28)
ReachOut Next Gen Community Workshop (n=16)
Project Folder Location:

34. Developing a framework that
articulates how participatory
design can be integrated with
evidence-based approaches.
Images from: Hagen, P, Collin, P, Metcalf, A, Nicholas, M, Rahilly, K, & Swainston, N 2012, Participatory Design of evidence-based online youth mental health promotion, prevention, early intervention and treatment, Young and Well Cooperative Research Centre, Melbourne.

35. Developing a framework that
articulates how participatory
design can be integrated with
evidence-based approaches.
Download from http://www.yawcrc.org.au/news/article/199#
Images from: Hagen, P, Collin, P, Metcalf, A, Nicholas, M, Rahilly, K, & Swainston, N 2012, Participatory Design of evidence-based online youth mental
health promotion, prevention, early intervention and treatment, Young and Well Cooperative Research Centre, Melbourne.

36. Supporting co-synthesis with
researchers & providing close
proximity to the actual stories,
quotes and experiences of
young people.
Should I go and see
the GP, even though
there is nothing
physically wrong??
My mobile is public.
Don’t SMS me
anything sensitive.
You can talk to me
at BigDay Out, but
not about anything
serious.
Oh, I’d delete the
app if it did that.

37. 03 our tools and
Evolving
approach to actually be more
rigorous

38. Methods and analysis are more
consistent with greater
transparency and
accountability.

39. Evolving our tools and
methods
#1. Adapting design tools to connect both perspectives
#2. Evolving how method and findings are communicated
#3. Being more critical about our tools and methods

40. Lessons Learned

41. Lesson 1.
There are inherent tensions between the two
approaches that may not be resolved. Neither is
a complete solution to the complexities and
challenges we face.

42. Lesson 2.
Both approaches should be challenged, and the
opportunity is in the new things we can build
out of this collaboration

43. Lesson 3.
Negotiating which perspective has greater
influence will be ongoing.
But how we gather and communicate our ‘data’
matters.

44. Lesson 4.
We bring lots of value but we need to communicate that
value and practice in a way that earns us the right to
have influence.

45. Lesson 5.
We are all learning.
Presentation by Kathryn McCabe a post-doc researcher on Work Out

46. In closing…
Opportunity
Marketing design as a
holistic approach
Opportunity =
Better integrating design
with other approaches
and perspectives

47. Being challenged is what helps
us to build a richer practice

48. Thank you.
Questions, for me or for
@mariesanicholas
@kittyrahilly