The DataTellAllSlides


Published on

Unlike other measures where clinical expertise or paying attention to nuance is not required (e.g, questionnaires that simply list symptoms and require Likert Scale responses), using the ORS in a clinically meaning way demands much, much more. It is an outcome measure that begins as a general snapshot of the client’s perspective of his or her life that evolves into an idiosyncratic, client specific measure of client progress defined by client described goal attainment. This presentation looks at how data can be used to not only insure data integrity but also identify clinicians who need additional support and training.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The DataTellAllSlides

  1. 1. 11/23/11 Data Integrity: How to Ensure Therapists Get It The Data Tell All Barry Duncan, Psy.D. Psy.D. 561.239.3640 1
  2. 2. 11/23/11 The ORS Is Not This The ORS Is Different Than Other Outcome Measures  Co-constructed with client  Goes from general look at client distress to specific representation of p client’s experience & reason for service  Requires nuance & skill in 2
  3. 3. 11/23/11 Co-Constructed With Client  To Be Relevant to Therapeutic Process  To Have Meaning from the Client’s Understanding of His or Her Life From General to Specific to Client’s Idiosyncratic Experience  At some point, p , connect the client’s described experience of their lives to the marks on the 3
  4. 4. 11/23/11 The ORS The Bare Bones  No specific content other than domains—a skeleton t th d i k l t to which clients add the flesh & blood of their experiences.  At the moment clients connect the marks with what they find distressing, the ORS becomes a meaningful measure of their progress and potent clinical tool. The ORS Is Different: Requires Nuance and Clinical Applications • Administering, But Don’t Get It. Clients must understand purpose (monitoring outcome, privileging th i perspective); Th i il i their ti ) Therapists must i t t understand same + make them meaningful • Administering, Using Some. But not the clinical cutoff or numbers…Heuristic clinical use but no continuity or coherence • Administering, Using Some. But not connecting to the client’s experience or reasons for service; got to get a “good” score; agency needs data with integrity • Administering the SRS. But seeing it as reflective of competence rather than an alliance building 4
  5. 5. 11/23/11 Collect Data from the Beginning  Used to think it could happen organically  What was I thinking!  Collecting data allows: knowing who is and g who isn’t doing it; who is and isn’t doing it correctly; and data integrity Supervision Just the Facts Ma’am • The data tell all: CDOI/PCOMS 5
  6. 6. 11/23/11 CDOI Supervision: Key to Client Benefit & Helper Growth • Based on outcome data not theory or pontification. • Aimed at early identification of clients at risk so services can be modified. Supervision: Four Steps • Start with graphs of all clients not reflecting benefit; also Includes scores in general • Supervisor looks for use of measures, over-utilization, data integrity, spends time on at risk clients, brainstorms options • Supervisor reviews stats & discusses ways to improve • Skill building, focus on client teachings, helper 6
  7. 7. 11/23/11 CDOI/PCOMS Supervision Data Integrity: What to Look For • 30% of Intakes over the Cutoff: • Client or therapist does not understand ORS –Role play introducing the ORS • ORS Scores between 35-40: • Client or therapist does not understand the measures; Rarely a good score; even mandated clients don’t score this high. Role play introducing the ORS, discussing score when it doesn’t match doesn t client description of life • ORS Scores Look Like a Saw • An emotional thermometer: Client or Therapist does not understand ORS— Role play connecting the client’s reason for service to the marks on the ORS. 30% of Intakes over the Cutoff Intake ORS • Client or Scores 1. 28.7 1 28 7 therapist 2. 17.5 3. 31.7 likely does not 4. 18.9 understand 5. 12.4 6. 29.6 ORS –Role 7. 33,6 play l 8. 7.6 9. 13.7 introducing 10.30.6 the 7
  8. 8. 11/23/11 The First Meeting Introducing the Measures  Introduce the ORS using your own i words—convey the notion of monitoring outcome and ensuring client voice is heard  Measure the marks and add the scores ORS Scores between 35-40 Intake ORS• Client or therapist does Scores not understand the 1. 28.7 1 28 7 measures; Rarely a 2. 17.5 good score; even 3. 36.7 4. 18.9 mandated clients don’t 5. 12.4 score this high. Role 6. 29.6 play introducing the 7. 38.6 ORS, discussing score ORS di i 8. 7.6 when it doesn’t match 9. 13.7 client description of 10.38.6 8
  9. 9. 11/23/11 The First Meeting Getting A Good Rating 1 1. Score ORS & give feedback re score & cutoff; 2. Client describes situation inconsistent w/score; 3. Practice checking it out with the client ORS Scores Look Like a Saw The Emotional Thermometer ORS Scores Client or Therapist 1. 28.7, 17.1, 26.4, 12.7, 29.9 1 28 7 17 1 26 4 12 7 29 9 2. 17.5, 19.6, 22.4 does not 3. 31.7, 12.2, 28.4, 6.7, understand ORS— 4. 18.9, 19.2, 19.7 5. 12.4, 18.3, 9.9, 21.1, 11.8 Role play 6. 29.6, 31.3 6 29 6 31 3 connecting the 7. 33,6, 9.3, 8.6, 34.9, 31.6, 3.4 client’s reason for 8. 7.6, 8.5, 9.5 9. 13.7, 14.5, 17.5, 20.8 service to the 10.30.6 marks on the 9
  10. 10. 11/23/11 Integrating the Measures Problems and Challenges?  Hasto be Relevant to the R l h Work  Or Becomes an Emotional Thermometer of Day to Day Life The First Meeting Integrating the Measures  1. Score ORS & give feedback re score & cutoff; 2. Connect marks to client description & reason for service; 3.Use examples from work & 10
  11. 11. 11/23/11 Reviewing Graphs Over Utilization • When max benefit reached, talk about stepping down; not pp g ; discharge, but planning for continued recovery out of tx. • If not stepped down, graphs look like a saw, rising and falling with everyday life; clients are di li t disempowered d • ORS represents life in general instead of clients perceptions connected to the purpose of the service. Reviewing Graphs Over Utilization Creates wait lists, iatrogenic effects. First way is when the client continues contin es in service in the absence of 11
  12. 12. 11/23/11 Supervisory Conversation The Longer w/o Change, the Quicker to #6 1.What does the client say? 2.Is the client engaged? SRS? 3.What has the helper done differently? 4.What can be done differently now? 5.What other resources can be rallied? 6.Is it time to fail successfully? Examining the Data Puts the Client in Charge  Place clients in position p of “super supervisor”  Ensures that supervisors stay on track and use clients’ voices to guide helpers in tough h l i t h circumstances.  Clients select who is discussed not 12
  13. 13. 11/23/11 Fidelity Monitoring Tool  Administer and score the Outcome Rating Scale (ORS) each visit or “unit of service ” unit service.  Ensure that the client understands that the ORS is intended to bring his or her voice into the decision-making process and will be collaboratively used to monitor progress.  Ensure that client gives a good rating; i.e., a rating that matches the client’s description.  Ensure that the client’s marks on the ORS are connected to the described reasons for service.  Use outcome (ORS) data to develop and graph individualized trajectories of change.  Plot client progress (ORS scores) on individualized trajectories from session to session to determine which clients are making progress and which are at risk for a negative or null outcome.  Use ORS scores to engage clients in a discussion in every session about how to y continue to empower change if it is happening and change, augment, or end tx if it is not.  Administer and score the Session Rating Scale (SRS) each visit or “unit of service.” 13
  14. 14. 11/23/11  Ensure that the client understands that the SRS is intended to create a dialogue between therapist and client that more tailors the service to the client and that there is no bad news on client--and the measure.  Use the SRS to discuss whether the client feels heard, understood, and respected.  Use the SRS to discuss whether the service is addressing the client’s goals for treatment.  Use the SRS to discuss whether the service approach matches the client’s culture or worldview, or theory of change. PCOMS Therapist Adherence Scale  Never Sometimes Often Regularly Always g y y  1 2 3 4 5  Out of a total possible 60 points, adherence is considered acceptable at 48 or above at the 6 month mark and 54 or above at the one year after implementation. High adherence is insured by the PCOMS supervisory 14