ReeseTolandSloneNorsworthy2010

Psychotherapy Theory, Research, Practice, Training © 2010 American Psychological Association2010, Vol. 47, No. 4, 616 – 630 0033-3204/10/$12.00 DOI: 10.1037/a0021182 EFFECT OF CLIENT FEEDBACK ON COUPLE PSYCHOTHERAPY OUTCOMES ROBERT J. REESE, LARRY A. NORSWORTHY MICHAEL D. TOLAND, AND Abilene Christian University NORAH C. SLONE University of KentuckyUsing outcome data to monitor the Keywords: couple therapy, patient-progress of treatment and the therapeu- focused research, feedback, psychother-tic alliance, also known as “client feed- apeutic outcomesback” or “patient-focused research,”has yielded impressive results in indi- Continuous assessment, or client feedback, is avidual psychotherapy. Client feedback method of tracking client progress across psycho-has demonstrated reductions in prema- therapy that allows for clinicians to monitorture terminations and improved psycho- whether progress is being made in treatment. If clients are not making progress as expected, ther-therapy outcomes. However, little re- apists have the opportunity to modify or adjustsearch has been conducted using this treatment as necessary. The American Psycholog-paradigm with couples receiving ther- ical Association (APA) Division 29 Task Force forapy. The purpose of this study was to Empirically Supported Relationships has made sev-investigate whether the effectiveness of eral recommendations to help clients achieve positive outcomes in psychotherapy. In a conclusionclient feedback would extend to couple based on these recommendations, Ackerman et al.therapy. Results from a randomized (2001) stated, “Practitioners are encouraged to rou-couple clinical trial conducted in a nat- tinely monitor patients’ responses to the therapyuralistic setting indicated that couples relationship and ongoing treatment. Such monitor-in a client feedback condition demon- ing leads to increasing opportunities to repair alliance ruptures, to improve the relationship, to mod-strated statistically significantly more ify technical strategies, and to avoid prematureimprovement compared with couples termination” (p. 496).receiving treatment as usual and that A growing body of research investigating theimprovement occurred more rapidly. effects of client feedback in psychotherapy has yielded encouraging results (e.g., Lambert et al.,Also, 4 times as many couples in the 2001, 2002; Reese, Norsworthy, & Rowlands,feedback condition reported clinically 2009; Whipple et al., 2003). A meta-analysis ofsignificant change by the end of treat- three previous client feedback studies conducted byment. Lambert and colleagues (2003) found an overall effect size (Cohen’s d) of 0.39 for clients in a feedback condition identified as deteriorating (de- clined in treatment by about half of a standard deviation) compared with clients deteriorating in a Robert J. Reese, Michael D. Toland, and Norah C. Slone, no-feedback condition. Client feedback has consis-Department of Educational, School, & Counseling Psychol- tently been found to benefit clients identified as atogy, University of Kentucky; and Larry A. Norsworthy, De-partment of Psychology, Abilene Christian University. risk for terminating prematurely (i.e., clients who do Correspondence regarding this article should be addressed not improve or deteriorate early in treatment).to Robert J. Reese, PhD, Department of Educational, School, The research, however, has found conflicting& Counseling Psychology, University of Kentucky, Lexing- results on whether client feedback works for allton, KY 40506-0017. E-mail: jeff.reese@uky.edu clients, not just those who are not progressing as616

Client Feedback in Couple Therapyexpected (Harmon et al., 2007; Lambert, Har- comes, therapists can more readily intervene andmon, Slade, Whipple, & Hawkins, 2005; Reese et attend to disconnects in the therapeutic relation-al., 2009). Conclusions are difficult because stud- ship. Harmon et al. (2007) and Whipple et al.ies that have found feedback to benefit all clients (2003) have found that adding clinical support(Harmon et al., 2007; Hawkins, Lambert, Ver- tools, including a measure of the therapeutic al-meersch, Slade, & Tuttle, 2004; Reese et al., liance, yielded incremental effectiveness for at-2009) have implemented client feedback differ- risk clients when compared with just trackingently from studies that have found improvement outcome.only for clients identified as at risk for terminat- The research on client feedback is impressiveing prematurely (Lambert et al., 2002; Whipple et but has focused almost exclusively on individualal., 2003). For example, Whipple et al. (2003) therapy. Couples and individuals experience sim-found that only clients not on track benefited ilar barriers to positive psychotherapy treatmentfrom feedback; however, the feedback data were outcome, including deterioration, premature ter-shared only with the therapist. In contrast, mination, and ruptured therapeutic alliancesHawkins et al. (2004) found that providing feed- (Snyder, Castellani, & Whisman, 2006). Meta-back was beneficial for all clients when it was analytic studies reported an overall effect sizeprovided to both the therapist and client. In turn, (Cohen’s d) for couple therapy ranging from 0.61Harmon et al. (2007) showed that feedback ben- (Shadish et al., 1993) to 0.84 (Shadish & Bald-efited all clients, but providing feedback to both win, 2003). According to recent findings (Snydertherapist and client did not lead to increased et al., 2006), couples that receive therapy areeffectiveness. More recent research (Reese et al., approximately 80% better off than couples that2009) has found that client feedback was benefi- do not receive treatment, which is comparable tocial for all clients when compared with a treat- effect sizes seen in the individual psychotherapyment as usual (TAU) condition. In this study, literature (Lambert & Ogles, 2004; Wampold,feedback was provided for both therapist and 2001). Shadish et al. (1993) compared the effectclient and included a measure to monitor the sizes for couple therapy studies with those fortherapeutic alliance every session. Research is individual therapy studies and found a nonsignif-needed to further address the processes by which icant difference between the effect sizes (dfeedback is most effective, but the continuous 0.05, SE 0.12, n 6). The outcome literatureassessment literature has consistently established provides substantial evidence that both individualthat feedback is beneficial for improving psycho- and couple psychotherapy are effective forms oftherapy outcomes, especially for clients at risk for treatment.dropping out of treatment. The psychotherapy outcome literature for cou- The rationale for continuous assessment is ples, like the individual literature, has also dem-based in part on research that has demonstrated onstrated that several approaches are effective.that, in the aggregate, clients who benefit from For example, Snyder et al. (2006) reported thattherapy demonstrate improvement sooner rather both emotion-focused couple therapy and behav-than later in treatment (e.g., Howard, Kopta, ioral couple therapy have yielded impressive re-Krause, & Orlinsky, 1986; Lutz, Martinovich, & sults in multiple clinical trials. Shadish and Bald-Howard, 1999). Monitoring outcome early in win’s (2005) meta-analytic findings suggest thattreatment increases the likelihood of identifying couples in treatment with behavioral couple ther-clients who are not progressing as expected. An- apy were 72% better off than couples in a controlother predictor of effective psychotherapy is hav- condition. Gollan and Jacobson (2002) demon-ing a strong therapeutic alliance (Horvath & strated effective couple therapy using emotion-Symonds, 1991). As was stated earlier, the APA focused couple therapy, demonstrating recoveryDivision 29 Task Force recommends the ongoing rates of 70 –73% with a weighted mean effectmonitoring of both outcome and the therapeutic size of 1.31 in comparison to a waitlist control.alliance. Monitoring the therapeutic alliance has Although these therapies have evidenced greaterbeen found to be a statistically significant predic- effectiveness when compared with no-treatmenttor of positive outcomes (Harmon et al., 2007). conditions, the literature suggests that when theseTherapists can quickly and directly respond to approaches are directly compared with one an-problems with the alliance when alerted (Lambert other, evidence has yet to demonstrate one ap-et al., 2002). As with monitoring treatment out- proach being superior to another (Shadish & 617

Reese, Toland, Slone, and NorsworthyBaldwin, 2003; Shadish et al., 1993; Snyder et back condition also reported higher levels of mar-al., 2006). ital satisfaction at posttreatment, and a greater One factor that makes studying couple therapy percentage of marriages were intact at follow-upoutcome difficult is that therapy is both an indi- when compared with marriages in the TAU con-vidual and shared experience for each partner. dition. These findings for PCOMS are consistentOutcome is affected by the influence of each with previous studies that focused on individualpartner’s readiness to change and level of distress therapy (Miller, Duncan, Brown, Sorrell, &(Isakson et al., 2006; Tambling & Johnson, Chalk, 2006; Reese et al., 2009).2008). An individual within the partnership may Replication of the Anker et al. (2009) findingshave significantly different views of the partner- is necessary for further evidence that continuousship itself, the therapy experience, and the ther- assessment, and specifically PCOMS, works withapist. A continuous feedback system used in cou- couples. The purpose of our study was to repli-ple therapy may allow both researchers and cate the results of Anker et al. with a sample fromclinicians to better understand how individuals the United States. The current study focused onrespond in couple psychotherapy at both the in- the effectiveness of using PCOMS with couplesdividual and couple level. in psychotherapy as compared with a TAU con- Only one study was identified that has used trol condition. We had three major hypotheses.client feedback with couples and therapists while First, we hypothesized that couples in the feedbackin therapy. Anker, Duncan, and Sparks (2009) condition would demonstrate better outcomes thanused the Partners for Change Outcome Manage- those in the TAU condition as measured by thement System (PCOMS; Duncan, Miller, & ORS after controlling for pre-ORS scores. Second,Sparks, 2004) with a sample of 205 White Euro- we hypothesized that couples in the feedback con-Scandinavian heterosexual couples. PCOMS condition would improve more quickly (i.e., in fewersists of two brief measures that are used to track sessions) than couples in the TAU condition. Third,client progress in therapy during each session. we hypothesized that more couples in the feedbackThe Outcome Rating Scale (ORS; Miller, Dun- condition would meet the criteria for clinical signif-can, Brown, Sparks, & Claud, 2003) consists of icance at posttreatment than would couples in thefour items and measures client outcome, and the TAU condition.Session Rating Scale (SRS; Duncan et al., 2003)also consists of four items and measures the ther-apeutic alliance. The ORS is administered at the Methodbeginning of each session and the SRS is admin- Participantsistered and scored at the end of each session. The results of each scale were administered, Clients. Clients were 46 heterosexual cou-scored, and discussed every session. Participants ples (N 92) that received couple therapy duringin the study presented with a broad range of the course of an academic year at a graduaterelationship issues, including communication, training clinic for a marriage and family therapyjealousy/conflict, and coping with partner’s phys- master’s program. There were 55 possible cou-ical or psychological problems (Anker et al., ples, but nine couples (3 feedback condition,2009). Couples were assigned to one of 10 ther- 6 TAU) did not return for a second session forapists and randomly assigned to a treatment con- reasons unknown. The mean pretreatment ORSdition: feedback or TAU. Couples in the feedback score for those that did not return for a secondcondition reported statistically significantly session (23.36) was almost identical to those in-higher residual ORS scores than couples in the cluded in the study (23.62). Seventy-four percentTAU condition, yielding an effect size of d of the sample was Caucasian (n 68), 4.3%0.50, which is considered large when comparing African American (n 4), 16.3% Hispanic/the differences between treatments (Wampold, Latino (n 15), 3.2% multiracial (n 3), and2001). Four times as many couples in the feed- 2.2% (n 2) did not indicate ethnicity. The meanback condition experienced clinically significant age was 30.18 years (SD 9.71), with ageschange (i.e., change beyond the standard error of ranging from 19 to 56 years. The primary reasonmeasure that includes starting treatment below couples sought counseling included relationshipthe clinical cut score and finishing treatment distress (n 36 couples; marital discord, com-above the clinical cut score). Couples in the feed- munication, parenting, divorce, separation, extra-618

Client Feedback in Couple Therapymarital affairs, sexual difficulties), individual dis- The internal consistency estimated with thetress affecting the relationship (n 4 couples; ORS (first session) for the current sample waspornography addiction, depression, anxiety, sex- .88, 95% CI [.84, .92]. Anker et al. (2009) re-ual abuse), and relationship enhancement (n 6 ported an internal consistency coefficient alphacouples; premarital, relationship enhancement). estimate of .93 with 410 individuals participating Therapists. All of the 261 sessions at the in couple therapy. Reese et al. (2009) have foundtraining clinic for marriage and family therapy similar reliability estimates. Evidence of concur-were provided by 13 second-year practicum stu- rent validity for scores derived from the ORS isdents (7 women and 6 men; 10 Caucasian, 2 based on Pearson correlations with scores onAfrican American, 1 Hispanic) enrolled in an other established outcome measures, includingAmerican Association for Marriage and Family the Symptom Checklist–90 —Revised (Deroga-Therapy–approved program. Practicum students tis, 1992; r .57; Reese, Norsworthy, & Row-received weekly individual and group supervi- lands, 2006), the Clinical Outcomes in Routinesion. All of the couple sessions were video- Evaluation (Barkham et al., 2001; r .67; Millerrecorded for supervision purposes. One therapist & Duncan, 2004), and the OQ45 (r .59; Millermet with the couples for a 50-min session typi- et al., 2003).cally on a weekly basis. There were no session PCOMS. All therapists and supervisors inlimits, and the sessions did not follow a particular the feedback condition attended a 1-hr trainingtreatment format or protocol. Theoretical orienta- session that covered the rationale for usingtions of the student therapists across both treatment PCOMS and how to administer, score, and inter-conditions were all grounded in a general family pret the ORS and SRS. The feedback conditionsystems framework, using a variety of approaches used the protocol as outlined in the scoring andincluding solution-focused, narrative/postmodern, administration manual for PCOMS (Miller &and strategic therapy. The median number of cou- Duncan, 2004). Each client was administered theples seen by each therapist was three, ranging from ORS at the beginning of every couple sessionone to eight couples. Therapists met with couples with the therapist present. After completing theon average for 5.91 sessions (Mdn 5), ranging ORS (approximately 1 min), the therapist scoredfrom two to 17 sessions. the items in the session. The total score was charted on a graph that indicated each client’s progress across treatment. Because this study wasMeasures conducted with couples, one chart was used that ORS. The ORS is a four-item, self-report showed the individual progress of each partner.measure that is designed to evaluate session-to- Therapists used the data within the session assession progress made in therapy. Using a visual they saw fit, but the manual provides guidelinesanalog scale, clients rate their level of psycholog- for how to intervene with clients who fall into theical distress on items adapted from the three areas following four categories:of the Outcomes Questionnaire 45 (OQ45; Lam-bert et al., 1996). Specifically, clients respond to ● No change. For a client who has not shownhow they are doing individually (personal well- reliable change (a gain of 5 points) after threebeing), socially (work, school, friendships), inter- sessions, therapists are directed to address thepersonally (family, close relationships), and therapeutic alliance and the course of treatment.overall (general sense of well-being). Clients If the client has not demonstrated reliable im-make a mark on each of the four analog scales provement after six sessions, the manual suggeststhat are 10 cm in length, with marks near the left consultation, supervision, or staffing.end of the scale indicating lower distress and ● Deteriorating. Clients in this category (a de-marks near the right end of the scale indicating crease of 5 points since entering treatment) arehigher distress. A ruler or template is then used to considered to be at risk for terminating prema-measure the distance from the left end of the turely or having a poor outcome. Therapists arescale to the client’s mark. The score is recorded directed to discuss possible reasons for thefor each item to the nearest millimeter and then drop in score, review the SRS items with theall are summed, for a total score ranging from 0 client to assess the therapeutic alliance, or con-to 40. Lower scores reflect more distress. sider changing the treatment approach, fre- 619

Reese, Toland, Slone, and Norsworthy quency, mode, or even therapist if no improve- nested wherein each client is nested within a ment is noted after three sessions. couple, which is then nested within a therapist.● Reliable change. Treatment is going accord- This means that the ORS scores of partners ingly (evidenced by a gain of at least 5 points within the same couple are likely to be more since beginning therapy). Therapists are ad- correlated than ORS scores for partners in differ- vised to reinforce changes and to continue ent couples. In the language of MLM, each client treatment until progress begins to plateau, is perceived as a Level-1 unit and couples are whereupon a therapist should consider reduc- seen as a Level-2 unit. Similarly, ORS scores of ing the frequency of sessions. couples within the same therapist are likely to be● Clinically significant change. The client may more correlated than ORS scores for couples no longer be struggling with issues that led to working with different therapists. As a result, the seeking therapy. Clinically significant change language of MLM would consider therapist to be is defined by a client beginning treatment be- a Level-3 unit. low the clinical cut score of 25, improving at For the first research hypothesis, we predicted least 5 points since starting therapy, and having that couples in the feedback condition would a total score in the nonclinical range (25 or demonstrate better outcomes than those in the above). Therapists are advised to consolidate TAU condition as measured by the ORS after changes, anticipate potential setbacks, and con- controlling for pre-ORS scores. This means a sider reducing the frequency of sessions. two-level cross-sectional multilevel model was needed to address this hypothesis. The Level-2 The SRS was administered to each client and predictor is feedback condition (FEEDBACK;again scored by the therapist (approximately 1 1 feedback condition; 0 TAU condition) andmin) toward the end of the session. If the total the Level-1 predictor or covariate is pre-ORSscore was below 36 or any one of the items was scores. Because the primary interest is in thebelow 9, the therapist followed up and asked Level-2 predictor, FEEDBACK, pre-ORS scoresabout the reason for the lower scores. The total were grand mean centered by subtracting eachscore was then charted on a graph for the corre- client’s score from the overall mean pre-ORSsponding session. Again, scores on the SRS for score (Mpre-ORS; for details on centering predic-each partner in the couple were recorded on one tors in MLM, see Enders & Tofighi, 2007).graph. The SRS was used as part of the feedback The multilevel model used to address the firstprocess for PCOMS, but the data were not in- research hypothesis or explain variation in ORScluded in the analyses for the current study. scores is Y ij 00 01 FEEDBACKjData Analysis 10 pre-ORSij Mpre-ORS 0j rij, (1) We applied multilevel modeling (MLM; Hox,2002), also referred to as hierarchical linear mod- where Yij is the post-ORS score for client i ineling (Raudenbush & Bryk, 2002), to answer the couple j; 00 is a fixed effect reflecting the overallfirst two primary hypotheses (for a gentle intro- mean post-ORS for couples in the TAU conditionduction to MLM, see Peugh, 2010). In general, after controlling for pre-ORS scores; 01 is amultilevel data tend to result when data are nat- fixed effect reflecting mean difference betweenurally nested data structures (e.g., clients nested couples in the TAU and feedback conditions afterwithin therapists, therapists nested within a coun- controlling for pre-ORS scores (i.e., a positive dif-seling center, repeated observations nested within ference would mean that couples in the feedbackclients, who are then nested within therapists). condition had a higher mean post-ORS than couplesThe issue with nested data structures is that the in the TAU condition after controlling for pre-ORStraditional assumption of independence of obser- scores); 01 is a fixed effect or covariate reflectingvations is violated, which is necessary for tradi- the slope between pre- and post-ORS scores aftertional techniques such as analysis of variance controlling for FEEDBACK; 0j is a Level-2 ran-(Peugh, 2010). Ignoring this issue will result in dom couple effect or the deviation of couple j frombiased parameter estimates (i.e., means, vari- the overall mean post-ORS for couples after con-ances, and covariances) and increase Type I error trolling for pre-ORS scores; and rij is a Level-1rates. In this study, the data structure is naturally random client effect or client ij’s difference in620

Client Feedback in Couple Therapypost-ORS score from the overall mean post-ORS in ORS for a one-session increment in time); 200for couples after controlling for pre-ORS scores. is the quadratic or curvature growth rate between In MLM, the random effects are estimated as adjacent sessions for those in the TAU condition 2 2variances such that Couple and Client capture the (i.e., the expected nonlinear change in ORS for aintercept variances in ORS scores at the couple one-session increment in time); 001 is the meanand client levels, respectively. Conceptually, difference between couples in the TAU and feed- 2 Couple measures the variation in mean post-ORS back conditions at the start of therapy; 101 is thescores across couples that is not due to feed- average linear slope difference between couplesback condition and is similar to MSBetween in in the TAU and feedback conditions (i.e., a pos-analysis of covariance (ANCOVA). Similarly, itive value would mean that couples in the feed- 2 Client is the average variance in individual back condition improved faster in ORS scoresclients’ scores within couples after accounting than those in the TAU condition); 201 is thefor pre-ORS scores and feedback condition and difference in curvature growth rates between cou-is like MSWithin in ANCOVA. ples in the TAU and feedback conditions (i.e., a For the second research hypothesis, we predicted positive value would mean that couples in thethat couples in the feedback condition would im- feedback condition have more positive curvatureprove more quickly (i.e., in fewer sessions) than growth rates than couples in the TAU condition);couples in the TAU condition. This means that a 00j is a Level-3 random couple effect or thethree-level multilevel growth model was needed to deviation of each couple mean from the overalladdress this hypothesis. In this model, repeated initial couple ORS mean; r0ij is a Level-2 randomobservations or time represented Level 1, which client effect or the deviation of client ij’s ORSare nested within each client (Level 2), which are score from the overall initial couple ORS mean;then nested within each couple (Level 3). The and etij is a Level-1 random client effect or clientLevel-3 predictor in this model is feedback con- ij’s residual error at session t (this error termdition (FEEDBACK; as previously defined) and reflects the difference between each client’s pre-Level-1 predictors are the time measure as a dicted and observed ORS score).linear function (SESSION) and nonlinear func- Similar to the random effects estimated in thetion (SESSION2). The nonlinear function of time multilevel model for the first hypothesis, the ran-allows the model to capture the curvature in the dom effects in growth models are estimated as 2 2 2ORS growth patterns, which is more realistic than variances such that Couple, Client, and Errorassuming all couples’ ORS scores grow in a estimate the intercept variances in ORS scores atlinear manner. the couple-level, client-level, and repeated obser- 2 The multilevel model used to address the sec- vations level, respectively. Conceptually, Coupleond research hypothesis or explain variation in measures the variance in mean ORS scores acrossORS scores over sessions is couples that is not due to feedback condition, 2 Client is the average variance in individual cli-Y tij 000 100 SESSIONtij ents’ scores within couples that is not due to 2 feedback condition (i.e., individual differences 200 SESSIONtij 001 FEEDBACKj 2 between clients), and Error captures within- 101 FEEDBACKj SESSIONtij person variation in ORS scores (i.e., the variabil- 2 ity of an individual client’s score around her or 201 FEEDBACKj SESSIONtij 00j his mean ORS score). r0ij etij, (2) For all MLM analyses, predictors or covariates were added to each of these basic models at thewhere Ytij is the ORS score at session t for client appropriate level (client level or couple level). Toi in couple j; 000 is a fixed effect reflecting the compare the statistical fit of competing models,overall average couple mean ORS at the start of we used the 2 log-likelihood (–2LL) value ortherapy for those in the TAU condition (centered deviance statistic from two nested models andat Session 1 such that substituting 0 for SESSION found the difference in deviance estimates. Mod-reflects the effect of the treatment at the first els are nested when one model is a subset of thesession); 100 is the overall average linear growth larger statistical model. Also, the deviance is arate between adjacent sessions for those in the measure of fit, and the higher the deviance, theTAU condition (i.e., the expected linear change poorer the fit of the model to the sample data. The 621

Reese, Toland, Slone, and Norsworthydifference in the deviances tests the null hypoth- in the TAU condition (see Table 1 for pre- andesis that two models do not have statistically post-ORS mean treatment scores, standard devi-significantly different model fits to the sample ations, and effect sizes within each condition). Todata. A rejection of this null hypothesis indicates evaluate the first hypothesis, we first estimated athat the model with more estimated parameters model like that shown in Equation 1 except wefits the sample data better than a model with included only the covariate pre-ORS scoresfewer estimated parameters. The difference in the (grand mean centered; Mpre-ORS 23.62) in thedeviance statistics is 2 distributed with degrees model (covariate-only model in Table 2). Theof freedom equal to the difference in parameters covariate-only model was estimated as a baselineestimated by two nested models. The test of two model as is typically done in traditional hierar-models’ deviances is often referred to in the sta- chal regression analyses to determine the incre-tistical literature as a likelihood ratio test. All mental improvement of one model to the next.MLM analyses were conducted with Proc Mixed The covariate-only model (see column 1 of Tablein SAS Version 9.2 using maximum likelihood 2) suggests a statistically significant positiveestimation and the Satterthwaite degrees of free- slope ( 10 0.26, p .001) between pre-ORSdom method. scores and post-ORS scores across clients. This means that scores improved from pre-ORS to post-ORS, while the average post-ORS for a cli-Results ent with an average pre-ORS score was 30.17Preliminary Analyses ( 00). The standardized mean effect size from pre- to post-ORS was 0.71 ([30.17 – 23.62]/ Although our data are inherently nested within 9.21]), indicating that clients improved by almosttherapists at the highest level, initial MLM anal- three fourths of a standard deviation from pre- toyses indicated that therapist fixed effects (i.e., post-ORS.feedback condition vs. TAU) could not be di- One way to understand the overall utility of therectly estimated at the therapist level because of covariate-only model is to compute a globalthe limited number of therapists used in this study pseudo-R2 effect size statistic, like multiple R2 in(n 13), but therapist-level variance at the in- regression. This is done by correlating and squar- 2tercept ( Therapist) could be estimated (i.e., the ing the predicted ORS scores for each participant,variation in mean ORS scores across therapists). using the fixed effects parameters for theIt is important to note that ignoring the variability covariate-only model with the observed ORSat the therapist level results in this variability scores for each client. The global pseudo-R2being pushed into the variance estimates at other .13, which means that 13% of the variation inlevels of the multilevel model, which are then ORS scores can be explained by knowing theultimately biased. Therefore, we estimated the pre-ORS scores (see Peugh, 2010). To under- 2 Therapist for each of our models. As a result, we stand the specific amount of variability explainedused a three-level multilevel model to analyze the at a level, we computed a local pseudo-r2 statis-nested structure of our data, clients nested within tic, which is similar to a semipartial r2 statistic incouples, to address the first research hypothesis. traditional regression. As such, the estimated pro-Similarly, we used a four-level multilevel growth portion of variance between couples explained bymodel (session within client within couple withintherapist). However, each of these models is es- TABLE 1. Pretest and Posttest Mean Outcome Rating Scaletimating a single variance component at the ther- (ORS) Scores and Effect Sizes for the Client Feedback andapist level and described as models with one less Treatment as Usual Conditionslevel (i.e., ignoring therapist fixed effects becausethey could not be estimated). Client Treatment feedback as usual (n 54) (n 38)Did PCOMS Produce Differences in Outcomes Measure M SD M SDfor Couples? Pre-ORS score 23.34 9.15 24.03 9.47 Descriptive statistics show that clients in the Post-ORS score 31.92 7.15 27.67 9.53 Standardized effect size 0.94 0.38feedback condition improved 8.58 points com-pared with the 3.64-point improvement by clients Note. Standardized effect size (Mpost – Mpre)/SDpre.622

Client Feedback in Couple Therapy TABLE 2. Fixed and Random Effect Estimates for Multilevel Models Predicting Postoutcome Rating Scale (ORS) Scores Parameter Covariate-only model ANCOVA Fixed effects (regression coefficients) Intercept: Mean post-ORS ( 00 ) 30.17 (0.99) 27.56 (1.46) Client pre-ORS ( 10 ) 0.26 (0.09) 0.27 (0.09) Feedback ( 01 ) 4.44 (1.9) Random effects (regression variances) 2 Client intercept variance ( Client) 33.27 (7.04) 33.48 (7.1) 2 Couple intercept variance ( Couple) 27.51 (10.31) 22.52 (9.35) 2 Therapist intercept variance ( Therapist) 1.14 0.99 Standardized effect size 0.71a 0.48b Note. Standard errors are in parentheses. Client pre-ORS client’s initial ORS score grand mean centered; Feedback type of feedback condition (0 treatment as usual; 1 feedback). a Standardized effect size (M post Mpre)/SDpre. 01 b Standardized effect size 2 2 nTAU 1 sTAU post ORS nFeedback 1 sFeedback post ORS . N 2 p .05. p .01. p .001.the covariate-only model with pre-ORS is 0.27 points higher than couples in the TAU condition(i.e., [37.63 – 27.51]/37.63). This means that 27% after controlling for pre-ORS scores (see lastof the between-couples variance in post-ORS column of Table 2). The standardized mean effectscores is accounted for by knowing the pre-ORS size between couples’ ORS scores after control-scores. Moreover, the intraclass correlation for ling for pre-ORS scores was 0.54 (see formula at 2 2 2couple (ICCCouple Couple/[ Client Couple bottom of Table 2; U.S. Department of Educa- 2 Therapist]) was .44 (which had been .53). This tion: Institute of Education Sciences, 2008, p.means that 44% of the variance in post-ORS 43). This means that the feedback conditionscores is due to pre-ORS scores. The ICCTherapist scored just over half a standard deviation higherwas .02, meaning that 2% of the variability in on post-ORS scores than the TAU condition afterpost-ORS scores (after controlling for pre-ORS controlling for pre-ORS scores, which is withinscores) was attributed to therapists, which is the range of other naturalistic therapist effectswithin the range of other naturalistic therapist (see Anker et al., 2009).effects (see Anker et al., 2009; Baldwin, Berkel- When we examine the ANCOVA model ran-jon, Atkins, Olsen, & Nielsen, 2009). dom effects, we see that 39.5% of the variance in For the second model we added treatment con- post-ORS scores remains between couples.dition (FEEDBACK) as a predictor to the former Moreover, comparing the ICCCouple from bothmodel (see Equation 1). The second model can be models (covariate only vs. ANCOVA), the pro-conceptually thought of as a multilevel ANCOVA portion of variance between couples explained bymodel (see column 2 of Table 2). The –2LLs for the the ANCOVA model is (27.51 – 22.52)/27.51covariate-only and ANCOVA models were 629.6 .18 or 18%. That is, 18% more between-couplesand 624.4, respectively. The difference in fit be- variance in post-ORS scores is explained bytween these two models was statistically signifi- knowing the type of feedback condition. Thecant, 2(1) 5.2, p .02. Results from this global pseudo-R2 effect size statistic for theapproach suggest that including a model with a ANCOVA model was .19.treatment effect for feedback while controllingfor pre-ORS scores improves the overall fit of the Preliminary Growth Curve Analysesmodel to the data versus including only pre-ORSscores. This means that couples in the feedback In the MLM literature, it is recommended thatcondition scored on average 4.44 ( 00) ORS a minimum of four time points be used to specify 623

Reese, Toland, Slone, and Norsworthya quadratic (or nonlinear) group model (Willett, p .001) and quadratic ( 100 0.08, pSinger, & Martin, 1998). By adding a nonlinear .001) growth rates (see Table 3). The globalcomponent to the model, researchers can increase pseudo-R2 effect size statistic for the uncondi-precision in estimates of change (Muthen, 1999; ´ tional model was .09, which suggests that 9% ofMuthen & Curran, 1997). However, couples var- ´ the variation in ORS scores can be explained byied in the number of sessions attended. Results knowing linear change and quadratic change.from preliminary analyses identified an outlier To evaluate whether growth rates vary be-couple (17 sessions) as an influential case and tween couples receiving TAU versus feedbackwas subsequently removed from all subsequent during couple therapy (Hypothesis 2; Equationmultilevel growth curve analyses. Although the 2), we added treatment condition (FEEDBACK)minimum recommended number of sessions for a to the growth model (conditional growth model;quadratic growth model is four, we chose to see Table 3). The –2LLs for the unconditionalinclude all couples with at least two sessions and conditional growth models were 2,684.8 andbecause models including all couples did not 2,675.9, respectively. The difference in fit betweendiffer from models including couples who at- these two models was statistically significant,tended a minimum of four sessions. Moreover, 2 (3) 8.9, p .053, indicating that couples in theincluding couples with fewer than four sessions feedback condition improved more quickly thanincreases the generality of the results to natural- couples in the TAU condition. The globalistic settings and helps maintain adequate statis- pseudo-R2 effect size statistic for the conditionaltical power. model was .10. Inspection of the conditional growth model results specifically shows that clients receiv-Growth Curves for Feedback and TAU ing feedback during couple therapy have a statisti-Conditions cally significant different linear growth rate com- To evaluate the second hypothesis, we first pared with those in the TAU condition ( 101 1.5,estimated a model like that shown in Equation 2 p .02, d 0.81, i.e., d [effect(time)]/SDpre,except that we did not include feedback condition where time 5 and SDpre 9.31; for more details,(FEEDBACK) in the growth model (see uncon- see Feingold, 2009, p. 7). Because the session num-ditional growth model in Table 2). The uncondi- bers varied across couples, the value of time was settional growth model estimates an intercept or to 5 to reflect the median number of sessions at-average starting mean ORS score, linear growth tended., However, there was not a statistically sig-rate, and nonlinear (quadratic) growth rate across nificant difference in the conditions’ quadraticcouples. These results indicate that all couples growth rates ( 201 0.11, p .14). A depictionstart with an average ORS score of 24.46 and of the difference in these growth rates up to fivehave statistically significant linear ( 100 1.87, sessions is presented in Figure 1. We chose to stop TABLE 3. Fixed and Random Effect Estimates for Multilevel Growth Models for Outcome Rating Scale (ORS) Scores Parameter Unconditional growth model Conditional growth modelFixed effects (regression coefficients) Intercept: Mean ORS ( 000 ) 24.46 (1.07) 24.53 (1.65) Session ( 100 ) 1.87 (0.28) 0.91 (0.56) Session2 ( 200 ) 0.08 (0.03) 0.01 (0.07) Feedback ( 001 ) 0.001 (2.16) Feedback Session ( 101 ) 1.5 (0.65) Feedback Session2 ( 201 ) 0.11 (0.07)Random effects (regression variances) 2 Error variance ( Error) 32.53 (2.62) 31.68 (2.55) 2 Client intercept variance ( Client) 10.01 (4.14) 10.3 (4.16) 2 Couple intercept variance ( Couple) 24.48 (9.82) 23.66 (9.68) 2 Therapist intercept variance ( Therapist) 10.7 10.78Note. Standard errors are in parentheses. Session session number centered at Session 1; Feedback type offeedback condition (0 treatment as usual; 1 feedback). p .05. p .01. p .001.624

Client Feedback in Couple Therapy 40 the client finishes therapy above an established 35 cut score that separates a clinical from nonclini-Predicted ORS 30 cal population. The cut score for the ORS is 25. 25 The reliable change index and cut score for the 20 ORS were based on two samples from a commu- 15 nity mental health center (Miller et al., 2003) and 10 a residential alcohol and drug treatment center 1 2 3 4 5 (Miller, Mee-Lee, Plum, & Hubble, 2005). Session More clients in the feedback condition, both at Feedback TAU the individual and couple level, completed treat-FIGURE 1. Average growth curves across five sessions for ment having obtained reliable (gain of 5 points)the feedback (dashed line) and treatment as usual (TAU; solid and clinically significant (gain of 5 points andline) conditions. crossing the clinical threshold) change when compared with clients in the TAU condition (seeat five sessions given that the sample median num- Table 4). Approximately 65% of clients at theber of sessions per couple was five. Figure 1 depicts individual level reported reliable or clinicalthe quicker improvement in ORS scores for the change in the feedback condition compared withfeedback condition over the TAU condition after approximately 31.6% in the TAU condition. Atfive sessions. the couple level (only couples where both partners met the criteria were included), 44.4% of theClinical Significance couples in the feedback condition compared with 15.8% of the couples in the TAU condition re- Observing the number of clients who incur ported reliable or clinically significant change.clinically significant change across treatment has Four times as many couples in the feedback con-become a common way to assess psychotherapy dition were categorized as obtaining clinicallyoutcome (Lambert, Hansen, & Bauer, 2008).Jacobson and Truax (1991) developed formulas significant change. It is important to note thatto evaluate change in therapy using the terms only 15 couples in the feedback condition and 11reliable change and clinically significant change couples in the TAU condition were eligible toto denote meaningful change in therapy. Reliable achieve clinical significance (both partners hadchange is simply the increase or decrease in a pre-ORS 25); if this is considered, then 53.3%client’s score on an outcome measure that ex- of the eligible couples in the feedback conditionceeds the measurement error for the instrument. achieved clinical significance and 18.2%For the ORS, the amount of change needed to achieved clinical significance in the TAU condi-incur reliable change is 5 or more points. A tion. Chi-square analyses indicated that the dif-decrease of 5 or more points is termed deterio- ferences in the outcome classifications acrossration. Clinically significant change occurs when treatment conditions were statistically significanta client has reliable change (gain of 5 points) and at both the individual, 2(3, N 92) 10.42, TABLE 4. Individuals and Couples That Achieved Clinical Significance or Reliable Change in the Client Feedback and Treatment as Usual Conditions Individuals Couples Client feedback Treatment as Client feedback Treatment as (n 54) usual (n 38) (n 27) usual (n 19) Classification n % n % n % n %1. Deteriorated 4 7.4 4 10.4 1 3.7 1 5.32. No change 15 27.8 22 57.9 3 11.1 9 47.43. Reliable change 9 16.7 2 5.3 4 14.8 1 5.34. Clinically significant change 26 48.1 10 26.3 8 29.6 2 10.55. Not classified 11 40.7 6 31.6Note. Couples were classified only if both partners completed treatment in the same category. 625

Reese, Toland, Slone, and Norsworthy 2p .02, and couple, (3, N 46) 8.18, p therapy achieved clinical significance. At the.04, levels. couple level, approximately 30% of the couples in the feedback condition that completed treat-Discussion ment were classified as clinically significant compared with only 10.5% in the TAU condition. An impressive amount of research has accu- These rates are much lower than those reportedmulated that supports the efficacy of using con- by Christensen et al. (2004), who found that 52%tinuous outcome assessment (i.e., client feed- of couples that received a form of behavior ther-back) in individual psychotherapy. Little research apy reported clinically significant change. Thishas been conducted to evaluate whether using comparison is problematic, however, on the sur-client feedback in couple therapy would yield face. The couples in the Christensen et al. studysimilar results. Our study investigated whether were chronic and more distressed and also at-the benefits of using a client feedback system, tended more sessions (22.9 sessions vs. 5.9 ses-PCOMS (Duncan et al., 2004), would extend to sions) than the couples in our sample. Whencouples in therapy. Results indicated that couples couples that were ineligible for clinical signifi-randomly assigned to a feedback condition expe- cance (pre-ORS scores 25) are removed fromrienced statistically significant more improve- our sample, the rate of achieving clinical signif-ment than those in the TAU condition and also icance is 53.3% for couples in the feedback con-improved more quickly as evidenced by a steeper dition, which is comparable to the Christensen etgrowth curve. Couples in the feedback condition al. study.were also more likely to incur reliable and clin- The differences between pre- and post-ORSically significant change. The results of this study scores for couples are similar to client feedbackare comparable with the studies that have used studies that used PCOMS compared with TAUPCOMS with individuals to address outcome for individual psychotherapy. Although the cur-(Miller et al., 2005; Reese et al., 2009) and with rent differences in outcome are slightly smallerthe only other couple therapy study using than the 10.8-points gain found in Miller et al.PCOMS (Anker et al., 2009). (2005) and the 12.69- and 10.83-point gains In our study, couples in the feedback condition found in two samples by Reese et al. (2009), theexperienced treatment gains more than double of difference in scores between treatment conditionsthose in the TAU condition on the ORS (8.58 (at least double) is similar.points vs. 3.64 points). Findings from Anker et al. Although the results of our study closely re-(2009) found almost identical treatment gains semble the Anker et al. (2009) couple study, therewhen comparing feedback and TAU conditions are two differences of note. First, the therapists in(8.3 points vs. 3.11 points). Effect sizes for the our sample were all graduate trainees and thefeedback condition in both studies were also therapists in the Anker et al. study were licensedfound to be large (d 0.8; Cohen, 1992). In professionals. Second, the trainees in our studyaddition, the effect size for the difference be- received much less training (1 hr vs. 8 hr). Thetween the feedback condition and TAU condition trainees in our study, however, did have supervi-(d 0.48) was also similar to the Anker et al. sors who were able to provide continued instruc-(2009) effect size (d 0.50). tion and discuss couple progress and the ORS and When observing treatment effectiveness from SRS measures. Our study provides evidence thata clinical significance perspective, clients in the client feedback is useful for therapy trainees whofeedback condition were more likely to experi- provide couple therapy.ence clinically significant change (48.1%) com-pared with those in the TAU condition (26.3%). Limitations of Our StudySimilar to the Anker et al. (2009) study, we foundthat 4 times as many couples in the feedback There are multiple limitations of our study thatcondition were classified as having incurred clin- warrant mentioning. First, many clients in theically significant change when compared with feedback condition had missing session data. Wethose in the TAU condition. The results for the were not able to discern a pattern for the missingfeedback condition are comparable to Shadish data other than from anecdotal evidence fromand Baldwin’s (2003) summary of meta-analytic therapists that indicated both logistical issues (“Istudies that reported 40 –50% of clients in couple forgot” or “I did not bring copies of the measures626

Client Feedback in Couple Therapyto the session”) and clinical reasons (“It did not were aware of PCOMS, and some expressed frus-feel necessary every week” or “The couple had a tration with not being able to use it with theircrisis and it did not feel appropriate to use”). All clients. It is possible that they may have beenclients had ORS and SRS data for at least half of applying components of the system verbally withtheir sessions, but the consequences of these their clients. This possibility, however, was notmissing data may have led to underestimating the monitored or evaluated.effects of the feedback intervention. Concerns for A fourth concern is that only therapist traineesthis limitation are tempered by the similar results were used, thereby limiting the generalizability ofin our study compared with those in the Anker et the results to therapists with more experience.al. (2009) study. Future research should investi- More experienced therapists may have usedgate the influence of administering a continuous PCOMS more effectively and demonstratedassessment system every session compared with larger treatment gains, or conversely, the lack ofevery second or third session. Such a study would experience may have heighted the demand char-address the potential differential effects for some acteristics and led to an overestimate of treatmentclinics that administer continuous assessment effects. For example, couples were aware thatsystems every few sessions rather than every their therapist was a trainee being evaluated andsession. perhaps did not want to negatively influence the A second limitation is that we did not use student’s grade. We do not believe, however, thatmultiple outcome measures, such as marital sat- this is a large concern. A previous study weisfaction or couple distress. We are also unable to conducted (Reese et al., 2009) did not show treat-extrapolate the results of our study to assume that ment outcome differences between licensed, pro-larger treatment gains in the feedback condition fessional staff and trainees. In addition, the Ankerresulted in more couples remaining together et al. (2009) couple study found similar treatmentwhen compared with the couples in the TAU outcomes with experienced therapists.condition. The findings of our study, however,are very similar to those in the Anker et al. (2009) Future Research and Conclusionsstudy, which provide evidence of the validity forthe ORS in couple therapy. Treatment gains as The use of PCOMS with both individuals andmeasured by the ORS in their study also showed couples appears to have much promise, but moretreatment gains on an established measure of research is needed to clarify the variables andmarital adjustment, the Locke–Wallace Marital mechanisms of change associated with the posi-Adjustment Test (Locke & Wallace, 1959). Cou- tive outcomes found in studies using PCOMS.ples who had better outcomes as measured by the There is little understanding of why PCOMSORS were also more likely to remain together at leads to better outcomes, and until these pro-a 6-month follow-up. cesses are better understood, the confidence one A third limitation is the lack of consistently can attribute to the specific effects of PCOMS ismonitored treatment integrity. The appealing limited. Continuous assessment originally wasqualities of PCOMS are that it is easy to imple- designed to identify clients not progressing asment and provides the therapist with latitude as expected. It would logically follow that beinghow to best integrate the measures into treatment. able to identity clients early in treatment who areAlthough conducting a study in a naturalistic not improving would afford the therapist the op-setting is a strength, a weakness is being uncer- portunity to alter treatment. However, PCOMStain of the differences in how PCOMS was im- has been found to work with all clients, includingplemented. The effects of feedback may have those progressing as expected. Most of the re-been underestimated. Anecdotally, therapists research using the OQ45 has found that continuousported differing levels of allegiance to using assessment is more beneficial for clients not pro-PCOMS. This concern is tempered by the fact gressing as expected (e.g., Whipple et al., 2003).that therapists received weekly supervision and PCOMS differs from Lambert and colleagues’supervisors were encouraged to ensure protocol (1996) signal system because PCOMS uses acompliance by noting the use of PCOMS in measure of the therapeutic relationship every ses-video-recorded sessions (all sessions were re- sion, whereas the Lambert et al. system uses ancorded) and through verbal reminders in supervi- alliance measure when a client is not improvingsion. Also, the therapists in the TAU condition as expected. Does this difference matter? The 627

Reese, Toland, Slone, and Norsworthyauthors of PCOMS (Duncan et al., 2004) have With the increased need to demonstrate psy-opined that having access to weekly feedback chotherapy’s utility due to forces such as man-regarding the relationship may serve to heighten aged care and third-party reimbursement, mea-attention and focus on the therapeutic alliance suring the progress of treatment as it occurs hasand promote active collaboration. Future research become an important area of study with excitingshould attempt to isolate the contribution of the results. Ongoing client feedback has been foundSRS to the effectiveness of PCOMS. to help avoid premature termination and meet the PCOMS may also be effective because seeing needs of clients in a more effective, efficientthe measures weekly creates expectancy effects manner. Overall, the results of this study indicateregarding improvement. A second possibility is that using client feedback is a useful approachthat cognitive dissonance plays a role in reporting with couples that received treatment at a graduateimproved outcome, and a good therapeutic rela- training clinic and are consistent with the findingstionship may increase the likelihood that clients from previous client feedback studies focused onfeel better and have a good therapeutic relation- individual therapy. More research needs to beship. Another possibility is that seeing one’s conducted, but PCOMS appears to hold muchgraphed progress promotes improvement; there is promise for use with couples given its ease of usean established body of literature in psychology and encouraging results.that points to the importance of receiving feed-back for promoting behavioral change (e.g., ReferencesAlvero, Bucklin, & Austin, 2001). UsingPCOMS may stimulate such behavioral changes ACKERMAN, S., SMITH, B., BEUTLER, L. E., GELSO, C. J.,for both clients and therapists. Studies that ma- GOLDFRIED, M. R., HILL, C., & RAINER, J. (2001).nipulate the manner in which therapist and client Empirically supported therapy relationships: Conclu- sions and recommendations of the Division 29 Taskobserve the data would help address these issues. Force. Psychotherapy Theory, Research, Practice,Similar studies have been conducted with the Training, 38, 495– 497. doi:10.1037/0033–3204.38.4.495OQ45 (e.g., Harmon et al., 2007; Hawkins et al., ALVERO, A. M., BUCKLIN, B. R., & AUSTIN, J. (2001). An2004), but as mentioned earlier, the findings have objective review of the effectiveness and essential characteristics of performance feedback in organizationalbeen mixed. settings. Journal of Organizational Behavior Manage- Future research should also investigate ment, 21, 3–29. doi:10.1300/J075v21n01_02whether demand characteristics influence scores ANKER, M. G., DUNCAN, B. L., & SPARKS, J. A. (2009).on the ORS and SRS given that the measures are Using client feedback to improve couple therapy out-completed and then directly discussed with the comes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77,therapist. The SRS, in particular, appears suscep- 693–704. doi:10.1037/a0016062tible to social desirability because the client is BALDWIN, S. A., BERKELJON, A., ATKINS, D. C., OLSEN,evaluating the quality of the session in the pres- J. A., & NIELSEN, S. L. (2009). Rates of change inence of the therapist. It is possible that the de- naturalistic psychotherapy: Contrasting dose-effect and good-enough level models of change. Journal of Con-mand characteristics are heighted in couple ther- sulting and Clinical Psychology, 77, 203–211. doi:apy given the added presence of a partner, which 10.1037/a0015235makes the results even more “public.” Perhaps BARKHAM, M., MARGISON, F., LEACH, C., LUCOCK, M.,simply the process of asking the client to evaluate MELLOR-CLARK, J., EVANS, C., & MCGRATH, G.the session is pivotal because it overtly commu- (2001). Service profiling and outcome benchmarking using the CORE-OM: Toward practice-based evidencenicates that the therapist values the client’s input in the psychological therapies. Journal of Consultingand the scores from the measure are less impor- and Clinical Psychology, 69, 184 –196. doi:10.1037/0022-tant. To evaluate the role of social desirability, a 006X.69.2.184study could have three randomized treatment CHRISTENSEN, A., ATKINS, D. C., BERNS, S., WHEELER, J., BAUCOM, D., & SIMPSON, L. E. (2004). Traditional ver-conditions use PCOMS. One condition would use sus integrative behavioral couple therapy for signifi-PCOMS as typically prescribed, the second con- cantly and chronically distressed married couples. Jour-dition would complete the SRS without the ther- nal of Consulting and Clinical Psychology, 72, 176 –191.apist present and see the results after the client doi:10.1037/0022-006X.72.2.176has left, and the third group would complete the COHEN, J. (1992). A power primer. Psychological Bulle- tin, 112, 155–159. doi: 10.1037/0033-2909.112.1.155SRS without the therapist having access to the DEROGATIS, L. R. (1992). Administration, scoring andresults. Differences in the scores and outcome procedures manual II. Baltimore: Clinical Psychometriccould be assessed. Research.628

Client Feedback in Couple TherapyDUNCAN, B. L., MILLER, S. D., REYNOLDS, L., SPARKS, J., 45.2. Stevenson, MD: American Professional Creden- CLAUD, D., BROWN, J., & JOHNSON, L. D. (2003). The tialing Services. Session Rating Scale: Psychometric properties of a “work- LAMBERT, M. J., HARMON, C., SLADE, K., WHIPPLE, J. L., ing” alliance scale. Journal of Brief Therapy, 3, 3–12. Re- & HAWKINS, E. J. (2005). Providing feedback to psy- trieved from http://www.journalbrieftherapy.com/ chotherapists on their patients’ progress: Clinical re-DUNCAN, B. L., MILLER, S. D., & SPARKS, J. A. (2004). sults and practice suggestions. Journal of Clinical Psy- The heroic client: A revolutionary way to improve effec- chology, 61, 165–174. doi:10.1002/jclp.20113 tiveness through client-directed, outcome-informed ther- LAMBERT, M. J., & OGLES, B. (2004). The efficacy and apy. San Francisco: Jossey-Bass. effectiveness of psychotherapy. In M. J. Lambert (Ed.),ENDERS, C. K., & TOFIGHI, D. (2007). Centering predic- Bergin and Garfield’s handbook of psychotherapy and tor variables in cross-sectional multilevel models: A behavior change (5th ed., pp. 139 –193). New York: Wiley. new look at an old issue. Psychological Methods, 12, LAMBERT, M. J., WHIPPLE, J. L., HAWKINS, E. J., VER- 121–138. doi:10.1037/1082-989X.12.2.121 MEERSCH, D. A., NIELSEN, S. L., & SMART, D. W.FEINGOLD, A. (2009). Effect sizes for growth-modeling (2003). Is it time for clinicians to routinely track patient analysis for controlled clinical trials in the same metric outcome? A meta-analysis. Clinical Psychology, 10, as for classical analysis. Psychological Methods, 14, 43– 288 –301. doi:10.1093/clipsy/bpg025 53. doi:10.1037/a0014699 LAMBERT, M. J., WHIPPLE, J. L., SMART, D. W., VER-GOLLAN, K. K., & JACOBSON, N. S. (2002). Developments MEERSCH, D. A., NIELSEN, S. L., & HAWKINS, E. J. in couple therapy research. In H. A. Liddle, D. A. (2001). The effects of providing therapists with feed- Santisteban, R. F. Levant, & J. H. Bray (Eds.), Family back on patient progress during psychotherapy: Are psychology: Science-based interventions (pp. 105–122). outcomes enhanced? Psychotherapy Research, 11, 49 – Washington, DC: American Psychological Association. 68. doi:10.1093/ptr/11.1.49H ARMON , C. S., L AMBERT , M. J., S MART , D. M., LAMBERT, M. J., WHIPPLE, J. L., VERMEERSCH, D. A., HAWKINS, E., NIELSEN, S. L., SLADE, K., & LUTZ, W. SMART, D. W., HAWKINS, E. J., NIELSON, S. L., & (2007). Enhancing outcome for potential treatment fail- GOATES, M. (2002). Enhancing psychotherapy out- ures: Therapist– client feedback and clinical support comes via providing feedback on client progress: A tools. Psychotherapy Research, 17, 379 –392. doi: replication. Clinical Psychology and Psychotherapy, 9, 10.1080/10503300600702331 91–103. doi:10.1002/cpp.324 LOCKE, H. J., & WALLACE, K. M. (1959). Short marital-HAWKINS, E. J., LAMBERT, M. J., VERMEERSCH, D. A., adjustment and prediction tests: Their reliability and SLADE, K. L., & TUTTLE, K. C. (2004). The therapeutic validity. Marriage and Family Living, 21, 251–255. Re- effects of providing patient progress information. Psy- trieved from http://www.jstor.org chotherapy Research, 14, 308 –327. doi:10.1093/ptr/ LUTZ, W., MARTINOVICH, Z., & HOWARD, K. (1999). kph027 Patient profiling: An application of random coefficientHORVATH, A. O., & SYMONDS, B. D. (1991). Relation regression models to depicting the response of a patient between working alliance and outcome in psychother- to outpatient psychotherapy. Journal of Consulting and apy: A meta-analysis. Journal of Counseling Psychol- Clinical Psychology, 67, 571–577. doi:10.1037/0022- ogy, 38, 139 –149. doi:10.1037/0022– 0167.38.2.139 006X.67.4.571HOWARD, K. L., KOPTA, S. M., KRAUSE, M. S., & MILLER, S. D., & DUNCAN, B. L. (2004). The Outcome ORLINKSY, D. E. (1986). The dose-effect relationship in and Session Rating Scales: Administration and scoring psychotherapy. American Psychologist, 41, 159 –164. manual. Ft. Lauderdale, FL: Author. doi:10.1037/0003-066X.41.2.159 MILLER, S. D., DUNCAN, B. L., BROWN, J., SORRELL, R., &HOX, J. (2002). Multilevel analysis techniques and appli- CHALK, B. (2006). Using outcome to inform and improve cations. Mahwah, NJ: Erlbaum. treatment outcomes. Journal of Brief Therapy, 5, 5–22.ISAKSON, R. L., HAWKINS, E. J., HARMON, S. C., SLADE, Retrieved from http://www.journalbrieftherapy.com/ K., MARTINEZ, J. S., & LAMBERT, M. J. (2006). Assess- MILLER, S. D., DUNCAN, B. L., BROWN, K., SPARKS, J., & ing couple therapy as a treatment for individual dis- CLAUD, D. (2003). The Outcome Rating Scale: A prelim- tress: When is referral to couple therapy contraindi- inary study of the reliability, validity, and feasibility of a cated? Contemporary Family Therapy, 28, 313–322. doi: brief visual analog measure. Journal of Brief Therapy, 2, 10.1007/s10591-006 –9008-9 91–100. Retrieved from http://www.journalbrieftherapyJACOBSON, N. S., & TRUAX, P. (1991). Clinical signifi- .com/ cance: A statistical approach to defining meaningful MILLER, S. D., MEE-LEE, D., PLUM, W., & HUBBLE, change in psychotherapy research. Journal of Consult- M. A. (2005). Making treatment count: Client-directed, ing and Clinical Psychology, 59, 12–19. doi:10.1037/ outcome informed clinical work with problem drinkers. 0022-006X.59.1.12 In J. L. Lebow (Ed.), Handbook of clinical family ther-LAMBERT, M. J., HANSEN, N. B., & BAUER, S. (2008). apy (pp. 281–308). Hoboken, NJ: Wiley. Assessing the clinical significance of outcome results. In MUTHEN, B. O. (1999, October 29). Re: Timepoints [Online ´ A. M. Nezu & C. M. Nezu (Eds.), Evidence-based discussion]. Retrieved from http://www.statmodel outcome research: A practical guide to conducting ran- .com/discussion/messages/14/20.html#POST16727 domized controlled trials for psychosocial interventions MUTHEN, B. O., & CURRAN, P. J. (1997). General longi- ´ (pp. 359 –378). New York: Oxford University Press. tudinal modeling of individual differences in experi-LAMBERT, M. J., HANSEN, N. B., UMPHRESS, V., LUNNEN, mental designs: A latent variable framework for anal- K., OKIISHI, J., BURLINGAME, G., & REISINGER, C. ysis and power estimation. Psychological Methods, 2, (1996). Administration and scoring manual for the OQ 371– 402. doi:10.1037/1082-989X.2.4.371 629

Reese, Toland, Slone, and NorsworthyPEUGH, J. L. (2010). A practical guide to multilevel mod- SNYDER, D. K., CASTELLANI, A. M., & WHISMAN, M. A. eling. Journal of School Psychology, 48, 85–112. doi: (2006). Current status and future directions in couple 10.1016/j.jsp.2009.09.002 therapy. Annual Review of Psychology, 57, 317–344.RAUDENBUSH, S. W., & BRYK, A. S. (2002). Hierarchical doi:10.1146/annurev.psych.56.091103.070154 linear models: Applications and data analysis methods TAMBLING, R. B., & JOHNSON, L. N. (2008). The relation- (6th ed.). Newbury Park, CA: Sage. ship between stages of change and outcome in coupleREESE, R. J., NORSWORTHY, L. A., & ROWLANDS, S. therapy. American Journal of Family Therapy, 36, 229 – (2006, August). Does a popular client feedback model 241. doi:10.1080/01926180701290941 improve psychotherapy outcome? Poster session pre- U.S. Department of Education: Institute of Education sented at the annual meeting of the American Psycho- Sciences. (2008). What works clearing house: Procedures logical Association, New Orleans, LA. and standards handbook (version 2.0). Retrieved fromREESE, R. J., NORSWORTHY, L., & ROWLANDS, S. (2009). http://ies.Ed.gov/ncee/wwc/pdf/wwc_procedures_ Does a continuous feedback model improve psycho- v2_standards_handbook.pdf therapy outcomes? Psychotherapy Theory, Research, WAMPOLD, B. E. (2001). The great psychotherapy debate: Practice, Training, 46, 418 – 431. doi:10.1037/a0017901 Models, methods, and ﬁndings. Hillsdale, NJ: Erlbaum.SHADISH, W. R., & BALDWIN, S. A. (2003). Meta-analysis WHIPPLE, J. L., LAMBERT, M. J., VERMEERSCH, D. A., of MFT interventions. Journal of Marital and Family SMART, D. W., NIELSEN, S. L., & HAWKINS, E. J. (2003). Therapy, 29, 547–570. doi:10.1111/j.1752– 0606 Improving the effects of psychotherapy: The use of .2003.tb01694.x early identiﬁcation of treatment failure and problem-SHADISH, W. R., & BALDWIN, S. A. (2005). Effects of solving strategies in routine practice. Journal of Coun- behavioral marital therapy: A meta-analysis of random- seling Psychology, 50, 59 – 68. doi:10.1037/0022– ized controlled trials. Journal of Consulting and Clinical 0167.50.1.59 Psychology, 73, 6 –14. doi:10.1037/0022-006X.73.1.6 WILLETT, J. B., SINGER, J. D., & MARTIN, N. C. (1998).SHADISH, W. R., MONTGOMERY, L., WILSON, P., WILSON, The design and analysis of longitudinal studies of de- M., BRIGHT, I., & OKWUMABUA, T. (1993). The effects velopment and psychopathology in context: Statistical of family and marital psychotherapies: A meta-analysis. models and methodological recommendations. Devel- Journal of Consulting and Clinical Psychology, 61, 992– opment and Psychopathology, 10, 395– 426. doi: 1002. doi:10.1037/0022-006X.61.6.992 10.1017/S0954579498001667630

http://heartandsoulofchange.com	281
https://heartandsoulofchange.com	19
http://heartandsoulofchange2.com	13
http://myprojecttest.com	6
http://www.slideshare.net	1
http://translate.googleusercontent.com	1

ReeseTolandSloneNorsworthy2010

by Barry Duncan

Accessibility

Categories

Upload Details

Usage Rights

6 Embeds 321

Statistics

ReeseTolandSloneNorsworthy2010 Document Transcript