Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Introduction Patient Reported Outcome Measures (PROMS)


Published on

An introduction to the key concepts of patient Reported Outcome Measures, including reliability and validity, generic versus disease specific,selection criteria and their adaptation for different cultural groups.

Published in: Health & Medicine

An Introduction Patient Reported Outcome Measures (PROMS)

  1. 1. An introduction to Patient Reported Outcome Measures (PROM’s) Dr Keith Meadows DHP Research & Consultancy Ltd July 2010
  2. 2. Overview <ul><li>PROM’s What are they and what are they used for? </li></ul><ul><li>Generic and condition specific - strengths and weaknesses </li></ul><ul><li>What should we look for when choosing a PROM </li></ul><ul><li>Adapting PROM’s for cultural and ethnic groups </li></ul><ul><li>Interpreting PROM data </li></ul>
  3. 3. Definitions <ul><li>Patient reported outcomes (PROM’s) </li></ul><ul><li>are outcomes known only to the patient </li></ul><ul><li>Patient reported outcome measures (PROM’s) are tools we use to measure patient outcomes </li></ul>
  4. 4. Variations on a theme <ul><li>Health status </li></ul><ul><li>Health-related quality of life (HRQoL) </li></ul><ul><li>Well-being </li></ul><ul><li>Health outcomes </li></ul><ul><li>Quality of life </li></ul><ul><li>Satisfaction </li></ul>
  5. 5. But all are based on… <ul><li>Self-assessment </li></ul>
  6. 6. Common content e.g. SF-36 <ul><li>Physical functioning </li></ul><ul><li>Role limitations due to physical health </li></ul><ul><li>Bodily pain </li></ul><ul><li>General health perceptions </li></ul><ul><li>Vitality </li></ul><ul><li>Social functioning </li></ul><ul><li>Role limitations due to emotional problems </li></ul><ul><li>Mental health </li></ul>
  7. 7. The underlying principle <ul><li>To measure across the complete continuum of a given construct </li></ul>No pain Severe pain
  8. 8. Something for everyone <ul><li>Patient choice </li></ul><ul><li>Audit </li></ul><ul><li>Quality improvement (Clinician & Provider) </li></ul><ul><li>Research </li></ul><ul><li>Training </li></ul><ul><li>After Black N. 2008 </li></ul>
  9. 9. General applications <ul><li>Measurement of the patient’s health status or health-related quality of life (HRQoL) at a single point in time </li></ul><ul><li>PROMs are used to derive measures of the outcomes of specific interventions. </li></ul><ul><li>Changes in health status at two different points in time (e.g. before and after an operation) can be used to derive a measure of the impact of health care interventions. </li></ul><ul><li>Certain PROMs suitable for purposes of economic evaluation (e.g., estimation of quality-adjusted life years – QALYs) </li></ul>
  10. 10. Some specific applications <ul><li>Personalised care planning </li></ul><ul><li>Self-assessment </li></ul><ul><li>Annual review </li></ul><ul><li>Informed decision making </li></ul><ul><li>Population health </li></ul>
  11. 11. Why consider the patient’s perspective? <ul><li>If quality is to be at the heart of everything we do , it must be understood from the perspective of the patient’ </li></ul><ul><li>‘ Just as important (as clinical measures) is the effectiveness of care from the patient’s own perspective which will be measured through patient-reported outcome measures’ </li></ul><ul><li>Next Stage Review 2008 </li></ul>
  12. 12. And also… <ul><li>Patients know better – ‘We know little of the clinical outcomes of NHS services from the patient’s perspective. PROM’s fill this gap ’ DH 2007 </li></ul><ul><li>Clinical outcomes not always related to how the patient feels </li></ul><ul><li>Patients like to be asked </li></ul>
  13. 13. What can PROM’s tell us? <ul><li>Which is the best treatment for the condition? </li></ul><ul><li>Is one subgroup of the population sicker than the others? </li></ul><ul><li>Is an individual patient’s condition getting better? </li></ul><ul><li>After Coulter A 2008 </li></ul>
  14. 14. What should we look for when choosing a PROM? …evidence that it is a measure of what it is supposed to measure
  15. 15. … .across the complete continuum of a given construct No limitations Mobility Severe limitations
  16. 16. … .and locating individuals by providing meaningful scores on that continuum Very anxious anxiety No anxiety
  17. 17. Therefore… <ul><li>It is important we choose the right PROM for our clinical practice, patients, study, trial </li></ul><ul><li>Check the information we get and don’t get from a PROM </li></ul>
  18. 18. The need for a conceptual model <ul><li>A diagram of proposed causal linkages among a set of concepts believed to be related to a particular disease (Earp & Enmett 1991). </li></ul><ul><li>A taxonomy of patient outcomes according to the underlying health concepts they represent and proposes specific and causal relationships between different health concepts (Wilson & Cleary 1995) </li></ul>
  19. 19. Diabetes Health Profile Conceptual Model © DHP Research & Consultancy Limited 2010 Impact Management Treatment Symptoms Diet Complications Diabetes Emotional Behavioural Anxiety Mood Limitations in Social/work functioning Eating behaviour DHP
  20. 20. Development based on…. <ul><li>Literature review </li></ul><ul><li>In-depth interviews with patients/clinicians family etc. </li></ul><ul><li>Thematic analysis of qualitative data </li></ul>
  21. 21. Conceptual framework <ul><li>A representation of the expected relationship of items within a domain within a PROM concept </li></ul><ul><li>Item A </li></ul><ul><li>Item B Domain score 1 </li></ul><ul><li>Item C </li></ul><ul><li>Item C </li></ul><ul><li>Item E Domain score 2 </li></ul><ul><li>Item F </li></ul>
  22. 22. We therefore must look for evidence of… <ul><li>Validity </li></ul><ul><li>Reliability </li></ul><ul><li>Responsiveness </li></ul><ul><li>Precision </li></ul>
  23. 23. Validity <ul><li>Does it measure what it is meant to? </li></ul><ul><li>Content validity - does the content reflect the concept/construct measured, is the content representative? </li></ul><ul><li>Face validity - do the individual items look as if they are measuring what they should? </li></ul><ul><li>Criterion validity - can the construct be measured accurately? </li></ul><ul><li>Construct validity - Is there a conceptual model or theoretical underpinning? </li></ul>
  24. 24. Reliability <ul><li>Are the results stable over time when applied to the same people at different time periods? (Test-retest reliability) </li></ul>
  25. 25. Precision <ul><li>Does the measure discriminate between different patient groups, health states, treatments etc? </li></ul>
  26. 26. Responsiveness <ul><li>Is the measure responsive to change when change is present? </li></ul>
  27. 27. Anything else? <ul><li>- Acceptability – will people fill it in? </li></ul><ul><li>Response rates </li></ul><ul><li>Item completion rates </li></ul><ul><li>Missing cases </li></ul>
  28. 28. <ul><li>Feasibility – how easy will it be to use? </li></ul><ul><li>Cost </li></ul><ul><li>Time </li></ul><ul><li>Ease of scoring </li></ul><ul><li>Interpreting scores </li></ul><ul><li>Supporting documentation (Manual, norm-reference scoring etc.) </li></ul>
  29. 29. Does the name of the PROM reflect what is being measured? <ul><li>Not all PROM’s have a conceptual model/theoretical underpinning </li></ul><ul><li>Some PROM’s are named by the items that are grouped together </li></ul>
  30. 30. Implications of choosing the wrong PROM <ul><li>Fail to identify significant outcomes </li></ul><ul><li>Mislead clinical practice </li></ul><ul><li>Misrepresent treatment and disease impact </li></ul><ul><li>After Cano S. 2008 </li></ul>
  31. 31. Generic and condition specific - strengths and weaknesses <ul><li>Generic </li></ul><ul><li>Suitable for the general population </li></ul><ul><li>Comparisons with other conditions/disease groups </li></ul><ul><li>Content may be redundant for certain condition/illnesses </li></ul><ul><li>Not sensitive to detecting disease-specific issues </li></ul><ul><li>Condition-specific </li></ul><ul><li>Specific to disease group </li></ul><ul><li>Sensitive to detecting clinically significant changes </li></ul><ul><li>Content relevant to target group </li></ul><ul><li>Cannot compare with general population </li></ul>
  32. 32. Disease-specific or generic? <ul><li>‘ Go for a combined approach?’ </li></ul>
  33. 33. Index v Multidimensional <ul><li>Overall score (but can be graded) </li></ul><ul><li>Less information </li></ul><ul><li>?Easier to score </li></ul><ul><li>Appropriate for cost-benefit analysis </li></ul><ul><li>Provides a profile </li></ul><ul><li>Reflects the important/different components of the illness </li></ul><ul><li>Provides more information </li></ul><ul><li>Can be long </li></ul>
  34. 34. Approaches to interpreting PROM data <ul><li>Minimally Important Difference (MID) to change </li></ul><ul><li>Known groups </li></ul><ul><li>Response to treatment </li></ul><ul><li>Normative and reference groups </li></ul><ul><li>Statistical significance </li></ul><ul><li>Effect size </li></ul>
  35. 35. Effect size <ul><li>Effect size is a simple way of quantifying the difference between two groups that has many advantages over the use of tests of statistical significance alone. Effect size emphasises the size of the difference rather than confounding this with sample size. </li></ul>≥ 0.8 Large 0.5 – 0.7 Moderate 0.2 - 0.4 Small
  36. 36. Using PROM’s for cultural and ethnic groups <ul><li>Technical problems </li></ul><ul><li>Conceptual problems </li></ul><ul><li>Linguistic problems </li></ul>
  37. 37. What is our aim? <ul><li>To achieve cross-cultural equivalence: </li></ul><ul><li>Semantic equivalence – equivalence in meaning of words </li></ul><ul><li>Conceptual equivalence – validity of the concepts in the target language </li></ul><ul><li>Idiomatic equivalence – equivalent idioms/expressions in target language </li></ul><ul><li>Experiential equivalence – situations should fit target language </li></ul>
  38. 38. Overview <ul><li>PROM’s must be fit for purpose </li></ul><ul><li>Are they valid and reliable with a clear conceptual/theoretical underpinning </li></ul><ul><li>Don’t choose PROM’s on basis of popularity alone </li></ul><ul><li>Take an evidence-base approach in selecting the PROM </li></ul><ul><li>Be sure what you want to measure </li></ul><ul><li>Don’t assume the PROM is accurate in telling you what it measures </li></ul><ul><li>Translation alone does not result in cultural equivalence </li></ul>
  39. 39. Overview cont’d <ul><li>Statistical significance does not mean clinical significance </li></ul><ul><li>p values can be misleading (large samples can result in high p values) </li></ul>
  40. 40. This presentation is a selection of slides taken from our training course ’Patient Reported Outcome Measures (PROMs) in Clinical Trials and Health Care – An introduction. If you would like more information on our training courses and or the Diabetes Health Profile email: Visit our website