The Importance of measuring outcomes, including Patient Reported Outcome Measures (PROMS) BAOT Lifelong Learning Event 10 November 2010 Dr Alison Laver-Fawcett Head of Programme, BHSC(Hons) Occupational Therapy York St John University

1. The Importance of measuring
outcomes, including Patient
Reported Outcome Measures
(PROMS)
BAOT Lifelong Learning Event
10 November 2010
Dr Alison Laver-Fawcett
Head of Programme, BHSC(Hons) Occupational Therapy
York St John University

2. The drive towards outcomes is
nothing new...
 Department of Health (DoH, 1998)
stated that the modernisation of care
“moves the focus away from who
provides the care, and places it firmly
on the quality of services experienced
by, and the outcomes achieved for,
individuals and their carers and
families” (para. 1.7).

3. Commissioning for
outcomes
 On the 1st
February 2010 I joined six
colleagues for a one day meeting to
contribute to the development of a
‘Commissioning for better outcomes in
Dementia Services’ workbook. The
workbook is being developed for
commissioners to assist the
implementation of the National
Dementia Strategy

4. Why measure outcomes?
Back in 2000 Dr. Carolyn Unsworth stated:
 ‘Current pressures to document outcomes
and demonstrate the efficacy of
occupational therapy intervention arise from
fiscal restraints as much as from the
humanitarian desire to provide the best
quality health care to consumers.’
 ‘However, measuring outcomes is important
in facilitating mutual goal setting, increasing
the focus of therapy on the client,
monitoring client progress, as well as
demonstrating that therapy is valuable’(p. 147)
 10 years on this still applies

5. • Disseminate results to stakeholders
• (client, carer, commissioner, manager)

6. Occupational Therapy
Process (COT, 2003)
1. referral or reason for
contact
2. information gathering
3. initial assessment
4. reason for intervention /
needs identification /
problem formulation
(defining desired outcome)
5. goal setting (negotiating
pre-determined outcome,
baseline measures)
6. action planning
7. action
8. ongoing assessment
and revision of action
9. outcome and outcome
measurement
(establishing actual
outcome)
10. end of intervention or
discharge
11. review (long-term
outcome)

7. Commissioning
Outcomes
 Service predetermined outcome(s):
these may be articulated in a service
specification, clinical guidelines, protocols
or standards. In a service commissioning
agreement PROMS may be mandated
 Individual client / staff outcome goals:
these are negotiated within the framework
of the above for each service user

8. Sources of outcome data
 Self-report: questionnaires, rating
scales or interviews given to the
service user to complete, including
PROMS
 Therapist observation: standardised
outcome measures or observations
related to SMART goals

9. Sources of outcome data
 Proxy-report: questionnaires or interviews given to
the informal care-giver (family member, friend) to
complete (may include PROMs)
 Proxy-report: outcome data provided by another
member of the MDT and / or other staff involved in
the care package / assessment / management
 Service data: discharge destination, length of stay,
waiting time, delayed transfers of care, cost of
packages of care, number, length and cost of
interventions

10. What are Patient Reported
Outcome Measures?
 Patient Reported Outcome Measures (PROMS)
employ short, self-completed questionnaires
which measure the patient’s health status or
health-related quality of life at a single point in
time.
 PROMs are used to derive measures of the
outcomes of specific interventions.
 Changes in health status measured by PROMs at
two different points in time (e.g. before and after
an operation) can be used to derive measure of
the impact of health care interventions.
(Department of Health, DH, 2009)

11. Patient-Reported Outcome
Measures (PROMs)
 ‘Patient-reported outcome measures
(PROMs) provide a means of gaining an
insight into the way patients perceive their
health and the impact that treatments or
adjustments to lifestyle have on their
quality of life. These instruments can be
completed by a patient or individual about
themselves, or by others on their behalf.’
(PROM Group, UHCE, University of Oxford)

12. Requirement to collect
PROMs
 The new Standard NHS Contract for
Acute Services, introduced in April
2008, included a requirement in
Schedule 5 for providers to report
from April 2009 on PROMs.
 This was for patients undergoing
Primary Unilateral Hip or Knee
replacements, Groin Hernia surgery or
surgery for Varicose Veins.

13. Context – what are the
drivers for PROMs?
 Lord Darzi’s Interim Report on the future of the NHS
recommended that patient-reported outcome measures
(PROMs) should have a greater role in the NHS (Darzi,
2007).
 Lord Darzi’s report ‘High Quality Care for All’ (2008) outlines
policy regarding payments to hospitals based on quality
measures as well as volume. These measures include
PROMs as a reflection of patients’ experiences and views.
 Guidance has now been issued regarding the routine
collection of PROMs for the selected elective procedures
(DH, 2009).
(Jenkinson, Gibbons and Fitzpatrick, 2009, p3)

14. The Patient Reported Outcome
Measurement Group in Oxford
 Patient-reported outcome measures (PROMs) offer
enormous potential to improve the quality and results
of health services.
 They provide validated evidence of health from the
point of view of the user or patient.
 They may be used to assess levels of health and need
in populations, and in users of services
 ...over time they can provide evidence of the outcomes
of services for the purposes of audit, quality assurance
and comparative performance evaluation.
 They may also improve the quality of interactions
between health professionals and individual service
users.
• (Jenkinson, Gibbons and Fitzpatrick, 2009, p3)

15. Generic and Condition Specific
PROMS
A range of PROMS are available:
 Generic PROMS are applicable to
and comparable across a range of
treatments or conditions,
 Condition-specific PROMS are
considered to ‘often be more sensitive
to changes in health status’ (DH,
2009)

16. Example of a Generic PROM
Euro-Qol EQ-5D
 This was identified as a generic PROM by Department of Health (2009)
and also as applicable for use with people with stroke (Jenkinson, Gibbons
and Fitzpatrick, 2009)
 EQ-5D is a standardised instrument for use as a measure of health
outcome
 Applicable to a wide range of health conditions and treatments, it
provides a simple descriptive profile and a single index value for health
status.
 EQ-5D is designed for self-completion by respondents and is ideally
suited for use in postal surveys, in clinics and face-to-face interviews. It
is cognitively simple, taking only a few minutes to complete.
Instructions to respondents are included in the questionnaire.
 For more information see: http://www.euroqol.org/

17. Examples of Condition
Specific PROMs
 Oxford Hip Score (Available from:
http://phi.uhce.ox.ac.uk/pdf/OxfordScores/O
)
 Oxford Knee Score (Available from:
http://phi.uhce.ox.ac.uk/pdf/OxfordScores/O
)

18. Examples of Condition
Specific PROMs
Stroke Impact Scale (SIS)
 The instrument content was derived from input
from stroke patients, caregivers and health
professionals with experience in the field of
stroke.
 It contains 59 items across eight domains
(strength, hand function, ADL/IADL, Mobility,
Emotion, Memory, Communication and Social
Participation).
 Jenkinson, Gibbons and Fitzpatrick (2009)
report that the SIS is an appropriate PROM
for people who have had a stroke, p24)

19. Some existing rehabilitation
measures are being identified
in reports related to PROMS
Nottingham Extended Activities of Daily Living (ADL)
Scale
 ‘The Nottingham Extended ADL Scale was developed
and evaluated as a questionnaire for postal use (Nouri
and Lincoln, 1987). It assesses the ability to carry out
functional tasks, such as using public transport,
housework, social life and hobbies. Scores in four
areas: mobility, kitchen tasks, domestic activities and
leisure activities can be added to give a summary
score out of 22. Respondents are asked whether they
do the activity rather than if they can do it, in order to
assess level of activity rather than capability.’
(Jenkinson, Gibbons and Fitzpatrick, 2009, p25)

20.  Bibliographic database comprising over 16000 records relating
to patient-reported outcome measures, with a specially
designed keyword search facility.
 Part of the NHS Information Centre for health & social care.
The most recent bibliographic update is current to April 2008.
 Systematic reviews of PROMs relevant to specific disease and
population (demographic) groups.
 General information on patient-reported health outcomes and
instrument selection, including guidance regarding the
selection of appropriate instruments for use in clinical trials,
clinical practice, and population surveys.
 http://phi.uhce.ox.ac.uk/home.php [accessed 1 March 2010]

21.  Links to related websites including
instruments, resources, organisations,
research groups, and journals.
 A page about the Oxford Orthopaedic
scores including PDFs of the
questionnaires and guides to their
usage.
 http://phi.uhce.ox.ac.uk/home.php
[accessed 1 March 2010]

22. Patient Reported Outcomes
Measurement Group
 Four new reports on PROMs for
specific conditions have been
published in 2010:
 Stroke
 Heart Failure
 Epilepsy
 Asthma
http://phi.uhce.ox.ac.uk/newpubs.php

23. Where to look for outcome
measures
 Asher IE (2007) Occupational
Therapy Assessment Tools: An
Annotated Index (3rd ed)
 Clarke C Sealey-Lapes C Kotsch L
(2001) Outcome measures
Information Pack for
Occupational Therapy
 COT (2009) SNOMED project list of
assessments

24. Link to SNOMED list of OT
assessments
 http://www.cot.co.uk/homepage/search_res
ults/?
SearchIT=3F67BEEA1FDF9FCFEA7C4779
A336E6CB (accessed 2 February 2010)
 This is a list of published assessments –
not all listed assessments have
psychometric data that supports their use
as measures of outcome
 We are now working to create a list of
outcome terms. Outcome measures have
been identified from the list of assessments
and are now being mapped to the ICF.

25. For mental health measures
 National Institute for Mental Health in
England (2008) Outcomes Compendium.
Helping you to select the right tool for
best mental health care practice in your
field. [on-line] Available from:
http://www.dh.gov.uk/prod_consum_dh/grou
ps/dh_digitalassets/documents/digitalasset/
dh_093677.pdf [Accessed 28.10.2010]

26. Where to look
Some text books include test critiques of
measures, for example:
 Law, Baum and Dunn (2001)
Measuring occupational
performance: supporting best
practice in occupational therapy.
Literature search:
Use your hospital librarian to help you
On-line: Pub-Med and Google Scholar

27. Commissioning for
outcomes
 If commissioners make evidence of
outcomes a requirement of a service
specification and contract – are you
ready to deliver evidence of the
effectiveness of your service?

28. Take home message:
 If we don’t define and measure
outcomes we don’t have evidence of
the worth of occupational therapy
services
 In a system where the driver is for
‘Commissioning for outcomes’ if we
can’t evidence our outcomes there is
a real risk that occupational therapy
will not be commissioned!

29. Web resources
 National Centre for Health Outcomes Development
http://nchod.uhce.ox.ac.uk/index.html [accessed 1
March 2010]
 National Institute for Mental Health in England
(2008) Outcome Compendium. Helping you to
select the right tool for best mental health care
practice in your field. [on-line] Available from:
http://www.dh.gov.uk/prod_consum_dh/groups/dh_d
igitalassets/documents/digitalasset/dh_093677.pdf
[Accessed 28.10.2010]

30. Reference:
Laver Fawcett AJ
2007
Principles of
Assessment and
Outcome
measurement for
Occupational
Therapists and
Physiotherapists.
Chichester:
John Wiley & Sons

31. References
 Asher IE (2007) Occupational Therapy Assessment Tools: An
Annotated Index (3rd ed) Bethesda: AOTA Press
 Clarke C Sealey-Lapes C Kotsch L (2001) Outcome measures
Information Pack for Occupational Therapy. London: College
of Occupational Therapists
 College of Occupational Therapists(2003) Occupational Therapy
defined as a complex intervention. London: College of
Occupational Therapists. Available from:
http://www.cot.org.uk/MainWebSite/Resources/Document/OTDefi
ned.pdf [accessed 26th January 2010]
 Corr S, Siddons L (2005) An Introduction to the Selection of
Outcome Measures. British Journal of Occupational Therapy,
68(5), 202-206

32. References
 Department of Health (DH, 2009) Guidance on the
Routine Collection of Patient Reported Outcome
Measures (PROMS) [on-line] Available from:
http://www.dh.gov.uk/prod_consum_dh/groups/dh_digit
alassets/@dh/@en/documents/digitalasset/dh_081179.
pdf [accessed 27.10.2010]
 Heaton J, Bamford C (2001) Assessing the Outcomes
of Equipment and Adaptations: Issues and
Approaches. British Journal of Occupational
Therapy, 64(7), p. 346-356

33. References
 Jenkinson C, Gibbons E, Fitzpatrick R (2009) A Structured
review of Patient Reported Outcome Measures related to
Stroke. Oxford, Patient-reported Outcome Measurement Group,
Department of Public Health, University of Oxford
 Law M, Baum C, Dunn W (2001) Measuring occupational
performance: supporting best practice in occupational
therapy. Thorofare: Slack
 Nouri FM, Lincoln NB (1987) An extended activities of daily living
index for stroke patients. Clinical Rehabilitation; 1: 301-305.
 Unsworth C (2000) Measuring the outcome of occupational
therapy: tools and resources. Australian Occupational Therapy
Journal 47: 147-158

34. Further reading
 Doble S E, Santha JC (2008). Occupational well-
being: Rethinking occupational therapy outcomes.
Canadian Journal of Occupational Therapy,
75(3), 184-190.
 James S, Corr S (2004) The Morriston Occupational
Therapy Outcome Measure (MOTOM): Measuring
What Matters. British Journal of Occupational
Therapy, 67(5): 210-216.
 Unsworth C (2005) Measuring Outcomes using the
Australian Therapy Outcome Measures for
Occupational Therapy (AusTOMs - OT): Data
Description and Tool Sensitivity . British Journal of
Occupational Therapy, 68(8), 354-366

35. Contact details
Alison J. Laver-Fawcett, PhD, O.T.(C), DipCOT
Head of Programme, Occupational Therapy
Faculty of Health and Life Sciences
York St John University
Lord Mayor’s Walk, York
YO31 7EX
01904-624624
a.laverfawcett@yorksj.ac.uk
www.yorksj.ac.uk

36. Research Centre for Occupation & Mental Health
(RCOMH)
Developing world class research in occupation and
mental health to influence best practice
www.yorksj.ac.uk/rcomh