The importance of_measuring_outcomes


Published on

The Importance of measuring outcomes, including Patient Reported Outcome Measures (PROMS)

BAOT Lifelong Learning Event
10 November 2010

Dr Alison Laver-Fawcett
Head of Programme, BHSC(Hons) Occupational Therapy
York St John University

Published in: Health & Medicine
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The importance of_measuring_outcomes

  1. 1. The Importance of measuring outcomes, including Patient Reported Outcome Measures (PROMS) BAOT Lifelong Learning Event 10 November 2010 Dr Alison Laver-Fawcett Head of Programme, BHSC(Hons) Occupational Therapy York St John University
  2. 2. The drive towards outcomes is nothing new...  Department of Health (DoH, 1998) stated that the modernisation of care “moves the focus away from who provides the care, and places it firmly on the quality of services experienced by, and the outcomes achieved for, individuals and their carers and families” (para. 1.7).
  3. 3. Commissioning for outcomes  On the 1st February 2010 I joined six colleagues for a one day meeting to contribute to the development of a ‘Commissioning for better outcomes in Dementia Services’ workbook. The workbook is being developed for commissioners to assist the implementation of the National Dementia Strategy
  4. 4. Why measure outcomes? Back in 2000 Dr. Carolyn Unsworth stated:  ‘Current pressures to document outcomes and demonstrate the efficacy of occupational therapy intervention arise from fiscal restraints as much as from the humanitarian desire to provide the best quality health care to consumers.’  ‘However, measuring outcomes is important in facilitating mutual goal setting, increasing the focus of therapy on the client, monitoring client progress, as well as demonstrating that therapy is valuable’(p. 147)  10 years on this still applies
  5. 5. • Disseminate results to stakeholders • (client, carer, commissioner, manager)
  6. 6. Occupational Therapy Process (COT, 2003) 1. referral or reason for contact 2. information gathering 3. initial assessment 4. reason for intervention / needs identification / problem formulation (defining desired outcome) 5. goal setting (negotiating pre-determined outcome, baseline measures) 6. action planning 7. action 8. ongoing assessment and revision of action 9. outcome and outcome measurement (establishing actual outcome) 10. end of intervention or discharge 11. review (long-term outcome)
  7. 7. Commissioning Outcomes  Service predetermined outcome(s): these may be articulated in a service specification, clinical guidelines, protocols or standards. In a service commissioning agreement PROMS may be mandated  Individual client / staff outcome goals: these are negotiated within the framework of the above for each service user
  8. 8. Sources of outcome data  Self-report: questionnaires, rating scales or interviews given to the service user to complete, including PROMS  Therapist observation: standardised outcome measures or observations related to SMART goals
  9. 9. Sources of outcome data  Proxy-report: questionnaires or interviews given to the informal care-giver (family member, friend) to complete (may include PROMs)  Proxy-report: outcome data provided by another member of the MDT and / or other staff involved in the care package / assessment / management  Service data: discharge destination, length of stay, waiting time, delayed transfers of care, cost of packages of care, number, length and cost of interventions
  10. 10. What are Patient Reported Outcome Measures?  Patient Reported Outcome Measures (PROMS) employ short, self-completed questionnaires which measure the patient’s health status or health-related quality of life at a single point in time.  PROMs are used to derive measures of the outcomes of specific interventions.  Changes in health status measured by PROMs at two different points in time (e.g. before and after an operation) can be used to derive measure of the impact of health care interventions. (Department of Health, DH, 2009)
  11. 11. Patient-Reported Outcome Measures (PROMs)  ‘Patient-reported outcome measures (PROMs) provide a means of gaining an insight into the way patients perceive their health and the impact that treatments or adjustments to lifestyle have on their quality of life. These instruments can be completed by a patient or individual about themselves, or by others on their behalf.’ (PROM Group, UHCE, University of Oxford)
  12. 12. Requirement to collect PROMs  The new Standard NHS Contract for Acute Services, introduced in April 2008, included a requirement in Schedule 5 for providers to report from April 2009 on PROMs.  This was for patients undergoing Primary Unilateral Hip or Knee replacements, Groin Hernia surgery or surgery for Varicose Veins.
  13. 13. Context – what are the drivers for PROMs?  Lord Darzi’s Interim Report on the future of the NHS recommended that patient-reported outcome measures (PROMs) should have a greater role in the NHS (Darzi, 2007).  Lord Darzi’s report ‘High Quality Care for All’ (2008) outlines policy regarding payments to hospitals based on quality measures as well as volume. These measures include PROMs as a reflection of patients’ experiences and views.  Guidance has now been issued regarding the routine collection of PROMs for the selected elective procedures (DH, 2009). (Jenkinson, Gibbons and Fitzpatrick, 2009, p3)
  14. 14. The Patient Reported Outcome Measurement Group in Oxford  Patient-reported outcome measures (PROMs) offer enormous potential to improve the quality and results of health services.  They provide validated evidence of health from the point of view of the user or patient.  They may be used to assess levels of health and need in populations, and in users of services  ...over time they can provide evidence of the outcomes of services for the purposes of audit, quality assurance and comparative performance evaluation.  They may also improve the quality of interactions between health professionals and individual service users. • (Jenkinson, Gibbons and Fitzpatrick, 2009, p3)
  15. 15. Generic and Condition Specific PROMS A range of PROMS are available:  Generic PROMS are applicable to and comparable across a range of treatments or conditions,  Condition-specific PROMS are considered to ‘often be more sensitive to changes in health status’ (DH, 2009)
  16. 16. Example of a Generic PROM Euro-Qol EQ-5D  This was identified as a generic PROM by Department of Health (2009) and also as applicable for use with people with stroke (Jenkinson, Gibbons and Fitzpatrick, 2009)  EQ-5D is a standardised instrument for use as a measure of health outcome  Applicable to a wide range of health conditions and treatments, it provides a simple descriptive profile and a single index value for health status.  EQ-5D is designed for self-completion by respondents and is ideally suited for use in postal surveys, in clinics and face-to-face interviews. It is cognitively simple, taking only a few minutes to complete. Instructions to respondents are included in the questionnaire.  For more information see:
  17. 17. Examples of Condition Specific PROMs  Oxford Hip Score (Available from: )  Oxford Knee Score (Available from: )
  18. 18. Examples of Condition Specific PROMs Stroke Impact Scale (SIS)  The instrument content was derived from input from stroke patients, caregivers and health professionals with experience in the field of stroke.  It contains 59 items across eight domains (strength, hand function, ADL/IADL, Mobility, Emotion, Memory, Communication and Social Participation).  Jenkinson, Gibbons and Fitzpatrick (2009) report that the SIS is an appropriate PROM for people who have had a stroke, p24)
  19. 19. Some existing rehabilitation measures are being identified in reports related to PROMS Nottingham Extended Activities of Daily Living (ADL) Scale  ‘The Nottingham Extended ADL Scale was developed and evaluated as a questionnaire for postal use (Nouri and Lincoln, 1987). It assesses the ability to carry out functional tasks, such as using public transport, housework, social life and hobbies. Scores in four areas: mobility, kitchen tasks, domestic activities and leisure activities can be added to give a summary score out of 22. Respondents are asked whether they do the activity rather than if they can do it, in order to assess level of activity rather than capability.’ (Jenkinson, Gibbons and Fitzpatrick, 2009, p25)
  20. 20.  Bibliographic database comprising over 16000 records relating to patient-reported outcome measures, with a specially designed keyword search facility.  Part of the NHS Information Centre for health & social care. The most recent bibliographic update is current to April 2008.  Systematic reviews of PROMs relevant to specific disease and population (demographic) groups.  General information on patient-reported health outcomes and instrument selection, including guidance regarding the selection of appropriate instruments for use in clinical trials, clinical practice, and population surveys.  [accessed 1 March 2010]
  21. 21.  Links to related websites including instruments, resources, organisations, research groups, and journals.  A page about the Oxford Orthopaedic scores including PDFs of the questionnaires and guides to their usage.  [accessed 1 March 2010]
  22. 22. Patient Reported Outcomes Measurement Group  Four new reports on PROMs for specific conditions have been published in 2010:  Stroke  Heart Failure  Epilepsy  Asthma
  23. 23. Where to look for outcome measures  Asher IE (2007) Occupational Therapy Assessment Tools: An Annotated Index (3rd ed)  Clarke C Sealey-Lapes C Kotsch L (2001) Outcome measures Information Pack for Occupational Therapy  COT (2009) SNOMED project list of assessments
  24. 24. Link to SNOMED list of OT assessments  ults/? SearchIT=3F67BEEA1FDF9FCFEA7C4779 A336E6CB (accessed 2 February 2010)  This is a list of published assessments – not all listed assessments have psychometric data that supports their use as measures of outcome  We are now working to create a list of outcome terms. Outcome measures have been identified from the list of assessments and are now being mapped to the ICF.
  25. 25. For mental health measures  National Institute for Mental Health in England (2008) Outcomes Compendium. Helping you to select the right tool for best mental health care practice in your field. [on-line] Available from: ps/dh_digitalassets/documents/digitalasset/ dh_093677.pdf [Accessed 28.10.2010]
  26. 26. Where to look Some text books include test critiques of measures, for example:  Law, Baum and Dunn (2001) Measuring occupational performance: supporting best practice in occupational therapy. Literature search: Use your hospital librarian to help you On-line: Pub-Med and Google Scholar
  27. 27. Commissioning for outcomes  If commissioners make evidence of outcomes a requirement of a service specification and contract – are you ready to deliver evidence of the effectiveness of your service?
  28. 28. Take home message:  If we don’t define and measure outcomes we don’t have evidence of the worth of occupational therapy services  In a system where the driver is for ‘Commissioning for outcomes’ if we can’t evidence our outcomes there is a real risk that occupational therapy will not be commissioned!
  29. 29. Web resources  National Centre for Health Outcomes Development [accessed 1 March 2010]  National Institute for Mental Health in England (2008) Outcome Compendium. Helping you to select the right tool for best mental health care practice in your field. [on-line] Available from: igitalassets/documents/digitalasset/dh_093677.pdf [Accessed 28.10.2010]
  30. 30. Reference: Laver Fawcett AJ 2007 Principles of Assessment and Outcome measurement for Occupational Therapists and Physiotherapists. Chichester: John Wiley & Sons
  31. 31. References  Asher IE (2007) Occupational Therapy Assessment Tools: An Annotated Index (3rd ed) Bethesda: AOTA Press  Clarke C Sealey-Lapes C Kotsch L (2001) Outcome measures Information Pack for Occupational Therapy. London: College of Occupational Therapists  College of Occupational Therapists(2003) Occupational Therapy defined as a complex intervention. London: College of Occupational Therapists. Available from: ned.pdf [accessed 26th January 2010]  Corr S, Siddons L (2005) An Introduction to the Selection of Outcome Measures. British Journal of Occupational Therapy, 68(5), 202-206
  32. 32. References  Department of Health (DH, 2009) Guidance on the Routine Collection of Patient Reported Outcome Measures (PROMS) [on-line] Available from: alassets/@dh/@en/documents/digitalasset/dh_081179. pdf [accessed 27.10.2010]  Heaton J, Bamford C (2001) Assessing the Outcomes of Equipment and Adaptations: Issues and Approaches. British Journal of Occupational Therapy, 64(7), p. 346-356
  33. 33. References  Jenkinson C, Gibbons E, Fitzpatrick R (2009) A Structured review of Patient Reported Outcome Measures related to Stroke. Oxford, Patient-reported Outcome Measurement Group, Department of Public Health, University of Oxford  Law M, Baum C, Dunn W (2001) Measuring occupational performance: supporting best practice in occupational therapy. Thorofare: Slack  Nouri FM, Lincoln NB (1987) An extended activities of daily living index for stroke patients. Clinical Rehabilitation; 1: 301-305.  Unsworth C (2000) Measuring the outcome of occupational therapy: tools and resources. Australian Occupational Therapy Journal 47: 147-158
  34. 34. Further reading  Doble S E, Santha JC (2008). Occupational well- being: Rethinking occupational therapy outcomes. Canadian Journal of Occupational Therapy, 75(3), 184-190.  James S, Corr S (2004) The Morriston Occupational Therapy Outcome Measure (MOTOM): Measuring What Matters. British Journal of Occupational Therapy, 67(5): 210-216.  Unsworth C (2005) Measuring Outcomes using the Australian Therapy Outcome Measures for Occupational Therapy (AusTOMs - OT): Data Description and Tool Sensitivity . British Journal of Occupational Therapy, 68(8), 354-366
  35. 35. Contact details Alison J. Laver-Fawcett, PhD, O.T.(C), DipCOT Head of Programme, Occupational Therapy Faculty of Health and Life Sciences York St John University Lord Mayor’s Walk, York YO31 7EX 01904-624624
  36. 36. Research Centre for Occupation & Mental Health (RCOMH) Developing world class research in occupation and mental health to influence best practice