Model of TreatmentEducation and its EvaluationProblem.docx

Model of Treatment/Education and its Evaluation
Problem(s)
Will Power +
Common Factors +
Any Specific Factor (any treatment model EBP or other) +
Feedback Informed Treatment + Deliberate Practice =
Effective Outcome
SPECIAL ARTICLE
How important are the common factors in
psychotherapy? An update
BRUCE E. WAMPOLD
Department of Counseling Psychology, University of
Wisconsin, Madison, WI, USA; Modum Bad Psychiatric Center,
Vikersund, Norway
The common factors have a long history in the field of
psychotherapy theory, research and practice. To understand the
evidence supporting
them as important therapeutic elements, the contextual model of
psychotherapy is outlined. Then the evidence, primarily from
meta-

analyses, is presented for particular common factors, including
alliance, empathy, expectations, cultural adaptation, and
therapist differ-
ences. Then the evidence for four factors related to specificity,
including treatment differences, specific ingredients, adherence,
and compe-
tence, is presented. The evidence supports the conclusion that
the common factors are important for producing the benefits of
psychotherapy.
Key words: Common factors, contextual model, psychotherapy,
alliance, empathy, expectations, cultural adaptation, therapist
differences,
specific ingredients
(World Psychiatry 2015;14:270–277)
The so-called common factors have a long history in psy-
chiatry, originating with a seminal article by S. Rosenzweig
in 1936 (1) and popularized by J. Frank in the various
editions of his book Persuasion and Healing (2-4). During
this period, the common factors have been both embraced
and dismissed, creating some tension (5-9). The purpose of
this paper is not to review or discuss the debate, but to pro-
vide an update, summarizing the evidence related to these
factors.
To understand the evidence for the common factors, it is
important to keep in mind that these factors are more than a
set of therapeutic elements that are common to all or most
psychotherapies. They collectively shape a theoretical mod-
el about the mechanisms of change in psychotherapy.
A particular common factor model, called the contextual
model, has been recently proposed (8,10). Although there
are other common factor models (e.g., 4,11), based on differ-

ent theoretical propositions, the predictions made about the
importance of various common factors are similar and the
choice of the model does not affect conclusions about the
impact of these factors. The contextual model is presented
below, followed by a review of the evidence for the common
factors imbedded in the model.
THE CONTEXTUAL MODEL
The contextual model posits that there are three path-
ways through which psychotherapy produces benefits. That
is, psychotherapy does not have a unitary influence on
patients, but rather works through various mechanisms.
The mechanisms underlying the three pathways entail
evolved characteristics of humans as the ultimate social spe-
cies; as such, psychotherapy is a special case of a social heal-
ing practice.
Thus, the contextual model provides an alternative expla-
nation for the benefits of psychotherapy to ones that empha-
size specific ingredients that are purportedly beneficial for
particular disorders due to remediation of an identifiable
deficit (8).
The three pathways of the contextual model involve: a)
the real relationship, b) the creation of expectations through
explanation of disorder and the treatment involved, and c)
the enactment of health promoting actions. Before these
pathways can be activated, an initial therapeutic relation-
ship must be established.
Initial therapeutic relationship
Before the work of therapy can begin, an initial bond
between therapist and patient needs to be created. E. Bordin

stated in 1979 that “some basic level of trust surely marks all
varieties of therapeutic relationships, but when attention is
directed toward the more protected recesses of inner experi-
ence, deeper bonds of trust and attachment are required
and developed” (12, p. 254). The initial meeting of patient
and therapist is essentially the meeting of two strangers,
with the patient making a determination of whether the
therapist is trustworthy, has the necessary expertise, and
will take the time and effort to understand both the problem
and the context in which the patient and the problem are
situated.
The formation of the initial bond is a combination of
bottom-up and top-down processing. Humans make very
rapid determination (within 100 ms), based on viewing the
face of another human, of whether the other person is trust-
worthy or not (13), suggesting that patients make very rapid
judgments about whether they can trust their therapist.
More than likely, patients make rapid judgments about the
dress of the therapist, the arrangement and decorations of
the room (e.g., diplomas on the wall), and other features of
the therapeutic setting (14). However, patients come to ther-
apy with expectations about the nature of psychotherapy as
270 World Psychiatry 14:3 - October 2015
well, due to prior experiences, recommendations of intimate
or influential others, cultural beliefs, and so forth. The initial
interaction between patient and therapist is critical, it seems,
because more patients prematurely terminate from therapy
after the first session than at any other point (15).
Pathway 1: The real relationship

The real relationship, defined psychodynamically, is “the
personal relationship between therapist and patient marked
by the extent to which each is genuine with the other and
perceives/experiences the other in ways that befit the other”
(16, p. 119). Although the psychotherapeutic relationship is
influenced by general social processes, it is an unusual social
relationship in that: a) the interaction is confidential, with
some statutory limits (e.g., child abuse reporting), and b) dis-
closure of difficult material (e.g., of infidelity to a spouse, of
shameful affect, and so forth) does not disrupt the social
bond. Indeed, in psychotherapy, the patient is able to talk
about difficult material without the threat that the therapist
will terminate the relationship.
The importance of human connection has been discussed
for decades, whether is it called attachment (17), belonging-
ness (18), social support (19), or the lack of loneliness
(20,21). In fact, perceived loneliness is a significant risk fac-
tor for mortality, equal to or exceeding smoking, obesity, not
exercising (for those with chronic cardiac disease or for
healthy individuals), environmental pollution, or excessive
drinking (22-24). Psychotherapy provides the patient a
human connection with an empathic and caring individual,
which should be health promoting, especially for patients
who have impoverished or chaotic social relations.
Pathway 2: Expectations
Research in a number of areas documents that expecta-
tions have a strong influence on experience (25). Indeed,
the purported price of a bottle of wine influences rating of
pleasantness as well as neural representations (26). The
burgeoning research on the effects of placebos documents
the importance of expectations, as placebos have robustly
shown to alter reported experience as well as demonstrating
physiological and neural mechanisms (27,28).

Expectations in psychotherapy work in several possible
ways. Frank (4) discussed how patients present to psycho-
therapy demoralized not only because of their distress, but
also because they have attempted many times and in many
ways to overcome their problems, always unsuccessfully.
Participating in psychotherapy appears to be a form of
remoralization.
However, therapy has more specific effects on expecta-
tions than simple remoralization. According to the contextu-
al model, patients come to therapy with an explanation for
their distress, formed from their own psychological beliefs,
which is sometimes called “folk psychology” (29-31). These
beliefs, which are influenced by cultural conceptualizations
of mental disorder but also are idiosyncratic, are typically not
adaptive, in the sense that they do not allow for solutions.
Psychotherapy provides an explanation for the patient’s diffi-
culties that is adaptive, in the sense that it provides a means
to overcome or cope with the difficulties. The patient comes
to believe that participating in and successfully completing
the tasks of therapy, whatever they may be, will be helpful in
coping with his or her problems, which then furthers for the
patient the expectation that he or she has ability to enact
what is needed. The belief that one can do what is necessary
to solve his or her problem has been discussed in various
ways, including discussions of mastery (4,32), self-efficacy
(33), or response expectancies (25).
Critical to the expectation pathway is that patients believe
that the explanation provided and the concomitant treat-
ment actions will be remedial for their problems. Conse-
quently, the patient and therapist will need to be in agree-
ment about the goals of therapy as well as the tasks, which
are two critical components of the therapeutic alliance

(34,35). Hatcher and Barends described the alliance as “the
degree to which the therapy dyad is engaged in collabora-
tive, purposive work” (36, p. 293). Creating expectations in
psychotherapy depends on a cogent theoretical explanation,
which is provided to the patient and which is accepted by
the patient, as well as on therapeutic activities that are con-
sistent with the explanation, and that the patient believes
will lead to control over his or her problems. A strong alli-
ance indicates that the patient accepts the treatment and is
working together with the therapist, creating confidence in
the patient that the treatment will be successful.
Pathway 3: Specific ingredients
The contextual model stipulates that there exists a treat-
ment, particularly one that the patient finds acceptable and
that he or she thinks will be remedial for his or her prob-
lems, creating the necessary expectations that the patient
will experience less distress. Every treatment that meets the
conditions of the contextual model will have specific ingre-
dients, that is, each cogent treatment contains certain well-
specified therapeutic actions.
The question is how the specific ingredients work to pro-
duce the benefits of psychotherapy. Advocates of specific
treatments argue that these ingredients are needed to reme-
diate a particular psychological deficit. The contextual
model posits that the specific ingredients not only create
expectations (pathway 2), but universally produce some
salubrious actions. That is, the therapist induces the patient
to enact some healthy actions, whether that may be thinking
about the world in less maladaptive ways and relying less on
dysfunctional schemas (cognitive-behavioral treatments),
improving interpersonal relations (interpersonal psycho-
therapy and some dynamic therapies), being more accepting

271
of one’s self (self-compassion therapies, acceptance and
commitment therapy), expressing difficult emotions (emo-
tion-focused and dynamic therapies), taking the perspective
of others (mentalization therapies), and so forth. The effect
of lifestyle variables on mental health has been understated
(37). A strong alliance is necessary for the third pathway as
well as the second, as without a strong collaborative work,
particularly agreement about the tasks of therapy, the
patient will not likely enact the healthy actions.
According to the contextual model, if the treatment elicits
healthy patient actions, it will be effective, whereas propo-
nents of specific ingredients as remedial for psychological
deficits predict that some treatments – those with the most
potent specific ingredients – will be more effective than
others (8).
EVIDENCE FOR VARIOUS COMMON FACTORS
Now that the contextual model has been briefly pre-
sented, attention is turned toward an update of the evidence
for the common factors. Each factor reviewed is imbedded
in the contextual model, although each of them is more
generically considered atheoretically as an important one.
As will be apparent, many of the common factors are not
theoretically or empirically distinct.
To present the evidence succinctly and with as little bias
and error as possible, we rely on meta-analyses of primary
studies. Studies that examine the association of levels of a
common factor and outcome are typically reported by some
type of correlation statistic (such as Pearson’s product-

moment correlation), whereas studies that experimentally
manipulate and compare conditions typically report some
standardized mean difference (such as Cohen’s d). For com-
parison purposes, correlational statistics are converted to
Cohen’s d. All meta-analyses reported aggregate statistics,
corrected for bias, based on the effects of individual studies
appropriated weighted. To understand the importance of
effects, Cohen (38) classified a d of 0.2 as small, 0.5 as medi-
um, and 0.8 as large. The evidence is summarized in Figure
1, where the effects of various common factors are com-
pared to those of various specific factors.
Alliance
The alliance is composed of three components: the bond,
the agreement about the goals of therapy, and the agreement
about the tasks of therapy (12). As discussed above, alliance
is a critical common factor, instrumental in both pathway 2
and pathway 3.
Alliance is the most researched common factor. Typically
the alliance is measured early in therapy (at session 3 or 4)
and correlated with final outcome. The most recent meta-
analysis of the alliance included nearly 200 studies involving
over 14,000 patients and found that the aggregate correla-
tion between alliance and outcome was about .27, which is
equivalent to a Cohen’s d of 0.57 (39), surpassing the thresh-
old for a medium sized effect.
There have been a number of criticisms of the conclusion
that alliance is an important factor in psychotherapy (40),
most of which have focused on the correlational nature of
alliance research. However, each of the criticisms has been
considered and has been found not to attenuate the impor-
tance of the alliance (see 8).

First, it could well be that early symptom relief causes a
strong alliance at the third or fourth session 2 that is, early
responders report better alliances and have better outcomes.
To address this threat, early therapy progress must be statisti-
cally controlled or longitudinal research is needed to examine
the association of alliance and symptoms over the course of
therapy. The studies that have examined this question have
found evidence to support either interpretation, but the better
designed and more sophisticated studies are converging on
the conclusion that the alliance predicts future change in
symptoms after controlling for already occurring change.
Second, it could be that the correlation between alliance
and outcome is due to the patients’ contributions to the alli-
ance. According to this line of thinking, some patients may
come to therapy well prepared to form a strong alliance and
it is these patients who also have a better prognosis, so the
alliance-outcome association is due to the characteristics of
the patients rather than to something that therapists provide
to the patients. Disentangling the patient and therapist con-
tributions involves the use of multilevel modeling. Recently,
Baldwin et al (41) performed such an analysis and found
that it was the therapist contribution which was important:
more effective therapists were able to form a strong alliance
across a range of patients. Patients’ contribution did not pre-
dict outcome: patients who are able to form better alliances,
perhaps because they have secure attachment histories, do
not have better prognoses. Indeed, patients with poor
attachment histories and chaotic interpersonal relation-
ships may well benefit from a therapist who is able to form
alliances with difficult patients. These results have been cor-
roborated by meta-analyses (42).
Third, there may be a halo effect if the patient rates both
the alliance and the outcome. However, meta-analyses have

shown that the alliance-outcome association is robust even
when alliance and outcome are rated by different people. It
also appears that the alliance is equally strong for cognitive-
behavioral therapies as it is for experiential or dynamic
treatments, whether a manual is used to guide treatment or
not, and whether the outcomes are targeted symptoms or
more global measures.
There are other threats to validity of the alliance as a potent
therapeutic factor, but the evidence for each of them is nonex-
istent or weak (8). The research evidence, by and large, sup-
ports the importance of the alliance as an important aspect of
psychotherapy, as predicted by the contextual model.
As mentioned above, distinctions between certain com-
mon factors are difficult to make. A distinction has been
made between the bond, as defined as a component of the
alliance, which is related to purposeful work, and the real
relationship, which is focused on the transference-free gen-
uine relationship (8,16). There is some evidence that the
real relationship is related to outcome, after controlling for
the alliance (16), and, although the evidence is not strong, it
does support the first pathway of the contextual model.
A second construct related for the alliance is labeled goal
consensus/collaboration. Although related to agreement
about the goals and tasks for therapy, goal consensus/
collaboration is measured with different instruments. As
shown in Figure 1, the effect for goal consensus and collabo-
ration is strong (d50.72), based on a meta-analysis of 15
studies (43).

Empathy and related constructs
Empathy, a complex process by which an individual
can be affected by and share the emotional state of anoth-
er, assess the reasons for another’s state, and identify with
the other by adopting his or her perspective, is thought to
be necessary for the cooperation, goal sharing, and regula-
tion of social interaction. Such capacities are critical to
infant and child rearing, as children, who are unable to
care for themselves, signal to the caregiver that care is
needed, a process that is then put to use to manage social
relations among communities of adult individuals. Thera-
pist expressed empathy is a primary common factor, criti-
cal to pathway 1 of the contextual model, but which also
augments the effect of expectations.
The power of the empathy in healing was beautifully
revealed in a study of placebo acupuncture for patients with
irritable bowel syndrome (44). Patients with this syndrome
were randomly assigned to a limited interaction condition,
an augmented relationship condition, or treatment as usual
(waiting list for acupuncture). In the limited interaction
condition, the acupuncturist met with the patient briefly,
but was not allowed to converse with him or her, and
administered the sham acupuncture (a device that gives the
sensation of having needles pierce the skin, but they do not).
In the augmented relationship condition, the practitioner
conversed with the patient about the symptoms, the rele-
vance of lifestyle and relationships to irritable bowel syn-
drome, as well as the patient’s understanding of the cause
and meaning of her disorder. All this was done in a warm
and friendly manner, using active listening, appropriate
silences for reflection, and a communication of confidence
and positive expectation. For the four dependent variables

(global improvement, adequate relief, symptom severity,
and quality of life), the two sham acupuncture conditions
were superior to treatment as usual. However, the augment-
ed relationship condition was superior to the limited inter-
action condition, particularly for quality of life.
The above study is noteworthy because it was an experi-
mental demonstration of the importance of a warm, caring,
empathic interaction within a healing setting. Unfortunate-
ly, experimental manipulation of empathy in psychotherapy
studies is not possible, for design and ethical reasons. None-
theless, there have been numerous studies (n559) that have
correlated ratings of therapist empathy with outcome,
which have been meta-analytically summarized (45), result-
ing in a relatively large effect (d50.63; see Figure 1). Con-
structs related to empathy have also been meta-analyzed
and found to be related to outcome, including positive
regard/affirmation (d50.56, n518; see Figure 1) (46) and
congruence/genuineness (d50.49, n518; see Figure 1) (47).
It should be recognized that several of the threats to validi-
ty for the alliance are also present with regard to empathy.
For example, it is clearly easier for a therapist to be warm and
Figure 1 Effect sizes for common factors of the contextual
model and specific factors. Width of bars is proportional to
number of studies on
which effect is based. RCTs – randomized controlled trials,
EBT – evidence-based treatments
273
caring toward a motivated, disclosing and cooperative pa-
tient than to one who is interpersonally aggressive, and the

former types of patients will most likely have better outcomes
than the latter, making the empathy/outcome correlation an
artifact of patient characteristics rather than therapist action.
Unfortunately, studies such as the ones conducted to rule out
these threats to validity for the alliance have not been con-
ducted for empathy and related constructs.
Expectations
Examining the role of expectations in psychotherapy is
difficult. In medicine, expectations can be induced verbally
and then physicochemical agents or procedures can be
administered or not, making the two components (creation
of expectations and the treatment) independent. In psycho-
therapy, creating the expectations, through explanation of
the patient’s disorder, presenting the rationale for the treat-
ment, and participating in the therapeutic actions, is part of
therapy. It is difficult to design experimental studies of
expectations in psychotherapy (not impossible, but not yet
accomplished in any important manner).
The typical way to assess the effect of expectations in psy-
chotherapy is to correlate patient ratings of their expecta-
tions with outcomes, but we have seen that such correla-
tional studies produce threats to validity. Furthermore, in
many studies, expectations are measured prior to when the
rationale for the treatment is provided to the patient, when
it is the explanation given to the patient that is supposed to
create the expectations. Assessing expectations after the
explanation has been given (i.e., during the course of treat-
ment) is also problematic, as those patients who have made
significant progress in therapy will naturally respond that
they think therapy will be helpful.
Despite the difficulties with investigating expectations in
psychotherapy, this is a topic of much interest (48-50).

Recently, a meta-analysis of expectations showed that there
was a relatively small, but statistically significant, relation-
ship between rated expectations and outcome (d50.24,
n546; see Figure 1) (49). The best evidence for expectations
in the context of healing is derived from studies of the place-
bo effect, where exquisite care has been taken to experimen-
tally manipulate variables of interest and to control for
threats to validity, by using physiological and neurological
variables as well as subjective reports. A summary of this lit-
erature is beyond the scope of this article, but many excel-
lent reviews are available (8,27,28).
Cultural adaptation of evidence-based treatments
The contextual model emphasizes that the explanation
given for the patient’s distress and the therapy actions must
be acceptable to the patient. Acceptance is partly a function
of consistency of the treatment with the patient’s beliefs, par-
ticularly beliefs about the nature of mental illness and how
to cope with the effects of the illness. This suggests that evi-
dence-based treatments that are culturally adapted will be
more effective for members of the cultural group for which
the adapted treatment is designed. There are many ways to
adapt treatments, including those involving language, cultur-
al congruence of therapist and patient, cultural rituals, and
explanations adapted to the “myth” of the group.
A recent meta-analysis demonstrated that adapting evi-
dence-based treatments by using an explanation congruent
with the cultural group’s beliefs (i.e., using the cultural
“myth” as the explanation) was more effective than unad-
apted evidence-based treatments, although the effect was
modest (d50.32, n521; see Figure 1) (51).
Therapist effects

Therapist effects are said to exist if some therapists consis-
tently achieve better outcomes with their patients than other
therapists, regardless of the nature of the patients or the
treatment delivered. Therapist effects have been studied in
clinical trials and in naturalistic settings. In both designs, the
measure of therapist effects is an intraclass correlation coef-
ficient. Technically, this coefficient indexes the degree to
which two patients from the same therapist have similar out-
comes relative to two patients from two different therapists.
To compare therapist effects to other common factors, the
intraclass correlation coefficient is converted to Cohen’s d.
The contextual model predicts that there will be differ-
ences among therapists within a treatment. That is, even
though the therapists are delivering the same specific ingre-
dients, some therapists will do so more skillfully and there-
fore achieve better outcomes than other therapists deliver-
ing the same treatment. Evidence for this conjecture is
found in clinical trials. A meta-analysis of therapist effects in
clinical trials found modest therapist effects (d50.35, n529;
see Figure 1) (52). Keep in mind that the therapists in clini-
cal trials generally are included because of their competence
and then they are given extra training, supervised, and mon-
itored. Moreover, the patients in such trials are homoge-
neous, as they have a designated diagnosis and are selected
based on various inclusionary/exclusionary criteria. In such
designs, patients are randomly assigned to therapists. Con-
sequently, consistent differences among therapists in such
trials, although modest, are instructive.
Not surprisingly, therapist effects in naturalistic settings
are greater than in clinical trials. In the former settings,
therapists are more heterogeneous, patients may not be ran-
domly assigned to therapists, patients are heterogeneous,
and so forth. A meta-analysis of therapist effects in such

settings found a relatively large effect (d50.55, n517; see
Figure 1) (52).
The finding of robust therapist effects raises the question
about what are the characteristics or actions of more effec-
tive therapists. Recent research has begun to address this
question. Studies have shown that effective therapists (vis-
!a-vis less effective therapists) are able to form stronger alli-
ances across a range of patients, have a greater level of facili-
tative interpersonal skills, express more professional self-
doubt, and engage in more time outside of the actual thera-
py practicing various therapy skills (8).
SPECIFIC EFFECTS
Evidence for the common factors is also collected by
examining the evidence for specific aspects of psychothera-
py. The contextual model makes several predictions about
specific effects, which will be discussed as each specific
effect is considered.
Treatment differences
When pathway 3 of the contextual model was discussed
earlier, it was emphasized that the model contends that all
therapies with structure, given by empathic and caring
therapists, and which facilitate the patient’s engagement in
behaviors that are salubrious, will have approximately equal
effects. That is, the specific ingredients, discussed in path-
way 3, are not critical because they remediate some psycho-
logical deficit.

The question of whether some treatments are superior to
others has long been debated, with origins at the very begin-
ning of the practice of psychotherapy (think about the dis-
agreements amongst Freud, Adler and Jung, for example).
Today, there are claims that some treatments, in general or
for specific disorders, are more effective than others. Others,
however, claim that there are no differences among psycho-
therapies, in terms of their outcomes.
The literature addressing this issue is immense and sum-
marizing the results of relative efficacy is not possible. Never-
theless, the various meta-analyses for psychotherapies in gen-
eral or for specific disorders, if they do find differences among
various types of treatment, typically find at most differences
of approximately d50.20, the value shown in Figure 1.
Specific effects from dismantling studies
To many, the dismantling design is the most valid way to
identify the effects of specific ingredients. In this design, a
specific ingredient is removed from a treatment to deter-
mine how much more effective the treatment is in total com-
pared to the treatment without the ingredient that is pur-
portedly remedial for the psychological deficit.
Two meta-analyses have examined dismantling designs
and both found minimal differences between the total treat-
ment and the treatment without one or more critical ingre-
dients (d50.01, n530, see Figure 1) (53,54). The most
recent of these meta-analyses did find that adding an ingre-
dient to an existing treatment increased the effect for tar-
geted variables by a small amount (d50.28) (53).
Adherence and competence

In clinical trials, it is required that adherence to the proto-
col and the competence at delivering the treatment are rat-
ed. This makes sense: if the goal is to make inferences about
a particular treatment, then it is necessary to ensure that the
treatment was delivered with the necessary components
and not with extraneous components (i.e., with adherence
to the protocol) and that the treatment components were
delivered skillfully (i.e., given competently).
It would seem logical theoretically that adherence to the
protocol and competence would be related to outcome.
That is, for cases where the therapist followed the protocol
and did so skillfully, there should be better outcomes. How-
ever, this is not the case. In a meta-analysis of adherence
and competence (55), effects were small (d50.04, n528 for
adherence; d50.14, n518 for competence; see Figure 1).
The results for adherence and competence demand fur-
ther explanation. If the specific ingredients of a treatment
are critical, then adherence should make a difference 2
actually delivering those ingredients should be related to
outcome. There is evidence that rigid adherence to a proto-
col can attenuate the alliance and increase resistance to the
treatment (i.e., failing to accept the treatment, a contextual
model tenet) (8), and that flexibility in adherence is related
to better outcomes (56), results consistent with prediction
of the contextual model.
The findings for competence are a bit more difficult to
understand. Competence in these trials typically is rated by
experts in the treatment being given, based on watching
therapy sessions. Why can’t experts differentiate between
“good” therapy and “bad” therapy? If this were indicative of
experts’ abilities to judge competence, then the notion of
psychotherapy supervision would be turned upside down,

because what is observed and evaluated would have no rela-
tion to outcomes 2 how could the supervisor then make a
case for providing input to the supervisee? But the clue to
the resolution of this mystery is found in the definition of
competence. Most psychotherapy trials rate the competence
for a specific treatment. That is, what is rated is the skill in
providing the elements of the treatment protocol, rather
than common factors, such as empathy, alliance, affirma-
tion, and so forth 2 aspects of therapy that do predict out-
come and seem to differentiate more effective therapists
from less effective therapists.
CONCLUSIONS
Although the common factors have been discussed for
almost a century, the focus of psychotherapy is typically on
the development and dissemination of treatment models. If
275
not discounted, then the common factors are thought of as
perhaps necessary, but clearly not sufficient. The evidence,
however, strongly suggests that the common factors must be
considered therapeutic and attention must be given to them,
in terms of theory, research and practice.
One of the criticisms of the common factors is that they
are an atheoretical collection of commonalities. In this
paper, the contextual model was presented to convey a the-
oretical basis for these factors.
References
1. Rosenzweig S. Some implicit common factors in diverse

methods
of psychotherapy: “At last the Dodo said, ‘Everybody has won
and all must have prizes’”. Am J Orthopsychiatry 1936;6:412-5.
2. Frank JD. Persuasion and healing: a comparative study of
psycho-
therapy. Baltimore: Johns Hopkins University Press, 1961.
3. Frank JD. Persuasion and healing: a comparative study of
psycho-
therapy, 2nd ed. Baltimore: Johns Hopkins University Press,
1973.
4. Frank JD, Frank JB. Persuasion and healing: a comparative
study
of psychotherapy, 3rd ed. Baltimore: Johns Hopkins University
Press, 1991.
5. Barlow DH. Psychological treatments. Am Psychol
2004;59:869-
78.
6. Laska KM, Gurman AS, Wampold BE. Expanding the lens of
evidence-based practice in psychotherapy: a common factors
per-
spective. Psychotherapy 2014;51:467-81.
7. Wampold BE. The great psychotherapy debate: model,
methods,
and findings. Mahwah: Lawrence Erlbaum Associates, 2001.
8. Wampold BE, Imel ZE. The great psychotherapy debate: the
research evidence for what works in psychotherapy, 2nd ed.
New
York: Routledge, 2015.

9. Baker TB, McFall RM, Shoham V. Current status and future
pros-
pects of clinical psychology: toward a scientifically principled
approach to mental and behavioral health care. Psychol Sci Publ
Int 2008;9:67-103.
10. Wampold BE, Budge SL. The 2011 Leona Tyler Award
address:
the relationship 2 and its relationship to the common and
specific
factors of psychotherapy. Couns Psychol 2012;40:601-23.
11. Orlinsky DE, Howard KI. Process and outcome in
psychothera-
py. In: Garfield SL, Bergin AE (eds). Handbook of
psychotherapy
and behavior change. New York: Wiley, 1986:311-81.
12. Bordin ES. The generalizability of the psychoanalytic
concept of
the working alliance. Psychotherapy: Theory, Research and
Prac-
tice 1979;16:252-60.
13. Willis J, Todorov A. First impressions: making up your
mind after
a 100-ms exposure to a face. Psychol Sci 2006;17:592-8.
14. Heppner PP, Claiborn CD. Social influence research in
counsel-
ing: a review and critique. J Couns Psychol 1989;36:365-87.
15. Connell J, Grant S, Mullin T. Client initiated termination of
thera-
py at NHS primary care counselling services. Couns Psychother
Res 2006;6:60-7.

16. Gelso C. A tripartite model of the therapeutic relationship:
theo-
ry, research, and practice. Psychother Res 2014;24:117-31.
17. Bowlby J. Attachment and loss. New York: Basic Books,
1980.
18. Baumeister RF. The cultural animal: human nature,
meaning, and
social life. New York: Oxford University Press, 2005.
19. Cohen S, Syme SL. Social support and health. San Diego:
Aca-
demic Press, 1985.
20. Cacioppo S, Cacioppo JT. Decoding the invisible forces of
social
connections. Front Integr Neurosci 2012;6:51.
21. Lieberman MD. Social: why our brains are wired to connect.
New York: Crown Publishing Group, 2013.
22. Holt-Lunstad J, Smith TB, Baker M et al. Loneliness and
social
isolation as risk factors for mortality: a meta-analytic review.
Per-
spect Psychol Sci 2015;10:227-37.
23. Holt-Lunstad J, Smith TB, Layton JB. Social relationships
and mor-
tality risk: a meta-analytic review. PLoS Med 2010;7:e1000316.
24. Luo Y, Hawkley LC, Waite LJ et al. Loneliness, health, and
mor-
tality in old age: a national longitudinal study. Soc Sci Med

2012;
74:907-14.
25. Kirsch I. How expectancies shape experience. Washington:
Ameri-
can Psychological Association, 1999.
26. Plassmann H, O’Doherty J, Shiv B et al. Marketing actions
can
modulate neural representations of experienced pleasantness.
Proc Natl Acad Sci USA 2008;105:1050-4.
27. Price DP, Finniss DG, Benedetti F. A comprehensive review
of
the placebo effect: recent advances and current thought. Annu
Rev Psychol 2008;59:565-90.
28. Benedetti F. Placebo effects: understanding the mechanisms
in
health and disease, 2nd ed. New York: Oxford University Press,
2014.
29. Boyer P, Barrett HC. Domain specificity and intuitive
ontologies.
In: Buss DM (ed). The handbook of evolutionary psychology.
Hoboken: Wiley, 2005:96-118.
30. Molden DC, Dweck CS. Finding ‘meaning’ in psychology: a
lay
theories approach to self-regulation, social perception, and
social
development. Am Psychol 2006;61:192-203.
31. Thomas RM. Folk psychologies across cultures. Thousand
Oaks:
Sage, 2001.

32. Liberman BL. The role of mastery in psychotherapy:
maintenance
of improvement and prescriptive change. In: Frank JD, Hoehn-
Saric R, Imber SD et al (eds). Effective ingredients of
successful
psychotherapy. New York: Brunner/Mazel, 1978:35-72.
33. Bandura A. Self-efficacy: toward a unifying theory of
behavioral
change. In: Baumeister RF (ed). The self in social psychology.
New York: Psychology Press, 1999:285-98.
34. Crits-Christoph P, Connolly Gibbons MB, Crits-Christoph K
et al. Can therapists be trained to improve their alliances? A
pre-
liminary study of alliance-fostering psychotherapy. Psychother
Res 2006;16:268-81.
35. Horvath AO. The alliance in context: accomplishments,
chal-
lenges, and future directions. Psychotherapy: Theory, Research,
Practice, Training 2006;43:258-63.
36. Hatcher RL, Barends AW. How a return to theory could help
alli-
ance research. Psychotherapy: Theory, Research, Practice,
Train-
ing 2006;43:292-9.
37. Walsh R. Lifestyle and mental health. Am Psychol
2011;66:579-
92.
38. Cohen J. Statistical power analysis for the behavioral
sciences,

2nd ed. Hillsdale: Erlbaum, 1988.
39. Horvath AO, Del Re AC, Fluckiger C et al. Alliance in
individual
psychotherapy. Psychotherapy 2011;48:9-16.
40. DeRubeis RJ, Brotman MA, Gibbons CJ. A conceptual and
meth-
odological analysis of the nonspecifics argument. Clin Psychol
Sci
Pract 2005;12:174-83.
41. Baldwin SA, Wampold BE, Imel ZE. Untangling the
alliance-
outcome correlation: exploring the relative importance of thera-
pist and patient variability in the alliance. J Consult Clin
Psychol
2007;75:842-52.
42. Del Re AC, Fl€uckiger C, Horvath AO et al. Therapist
effects in the
therapeutic alliance-outcome relationship: a restricted-maximum
likelihood meta-analysis. Clin Psychol Rev 2012;32:642-9.
43. Tryon GS, Winograd G. Goal consensus and collaboration.
In:
Norcross JC (ed). Psychotherapy relationships that work: evi-
dence-based responsiveness, 2nd ed. New York: Oxford
Universi-
ty Press, 2011:153-67.
44. Kaptchuk TJ, Kelley JM, Conboy LA et al. Components of
placebo
effect: randomised controlled trial in patients with irritable
bowel
syndrome. BMJ 2008;336:999-1003.

45. Elliott R, Bohart AC, Watson JC et al. Empathy.
Psychotherapy
2011;48:43-9.
46. Farber BA, Doolin EM. Positive regard. Psychotherapy
2011;48:58-
64.
47. Kolden GG, Klein MH, Wang CC et al.
Congruence/genuineness.
Psychotherapy 2011;48:65-71.
48. Greenberg RP, Constantino MJ, Bruce N. Are patient
expecta-
tions still relevant for psychotherapy process and outcome? Clin
Psychol Rev 2006;26:657-78.
49. Constantino MJ, Arnkoff DB, Glass CR et al. Expectations.
J Clin
Psychol 2011;67:184-92.
50. Arnkoff DB, Glass CR, Shapiro SJ. Expectations and
preferences.
In: Norcross JC (ed). Psychotherapy relationships that work:
therapist contributions and responsiveness to patients. Oxford:
Oxford University Press, 2002:335-56.
51. Benish SG, Quintana S, Wampold BE. Culturally adapted
psy-
chotherapy and the legitimacy of myth: a direct-comparison
meta-analysis. J Couns Psychol 2011;58:279-89.

52. Baldwin SA, Imel ZE. Therapist effects: finding and
methods.
In: Lambert MJ (ed). Bergin and Garfield’s handbook of psy-
chotherapy and behavior change. New York: Wiley, 2013:258-
97.
53. Bell EC, Marcus DK, Goodlad JK. Are the parts as good as
the
whole? A meta-analysis of component treatment studies. J Con-
sult Clin Psychol 2013;81:722-36.
54. Ahn H, Wampold BE. Where oh where are the specific
ingre-
dients? A meta-analysis of component studies in counseling and
psychotherapy. J Couns Psychol 2001;48:251-7.
55. Webb CA, DeRubeis RJ, Barber JP. Therapist
adherence/compe-
tence and treatment outcome: a meta-analytic review. J Consult
Clin Psychol 2010;78:200-11.
56. Owen J, Hilsenroth MJ. Treatment adherence: the
importance of
therapist flexibility in relation to therapy outcomes. J Couns
Psy-
chol 2014;61:280-8.
DOI 10.1002/wps.20238
277
What Works in Therapy:

A Primer
ICCE Manuals on Feedback-Informed Treatment (FIT)
Manual 1
© 2012, International Center for Clinical Excellence
The material in this manual is copyrighted and protected by all
laws and statutes, local and international, regarding copyrighted
material. No part of this volume may be copied, quoted, or
transcribed, in whole or in part, electronically or otherwise,
without
written permission. The volume and contents are intended for
use by the purchaser alone and may not be forwarded, attached,
or appended, in whole or in part, electronically or otherwise,
without written permission of the publisher.
The ICCE Manuals on Feedback-Informed Treatment (FIT)
Scott D. Miller, Co-Founder, ICCE
Bob Bertolino and Scott D. Miller, Series Editors for ICCE
Manuals
The ICCE Manuals on FIT were a collaborative effort. The
development team included:
Rob Axsen, Susanne Bargmann, Robbie Babbins-Wagner, Bob
Bertolino, Cynthia
Maeschalck, Scott D. Miller, Bill Robinson, Jason Seidel, and
Julie Tilsen.

Manual Authors:
Manual 1: What Works in Therapy: A Primer
Bob Bertolino, Susanne Bargmann, Scott D. Miller
Manual 2: Feedback-Informed Clinical Work: The Basics
Susanne Bargmann, Bill Robinson
Manual 3: Feedback-Informed Supervision
Cynthia Maeschalck, Susanne Bargmann, Scott D. Miller, Bob
Bertolino
Manual 4: Documenting Change: A Primer on Measurement,
Analysis, and Reporting
Jason Seidel, Scott D. Miller
Manual 5: Feedback-Informed Clinical Work: specific
populations
and service settings
Julie Tilsen, Cynthia Maeschalck, Jason Seidel, Bill Robinson,
Scott D. Miller
Manual 6: Implementing Feedback-Informed Work in Agencies
and Systems of Care
Bob Bertolino, Rob Axsen, Cynthia Maeschalck, Scott D.
Miller,
Robbie Babbins-Wagner

What Works in Therapy: A Primer
1
Introduction to the Series of Manuals
The International Center for Clinical Excellence (ICCE)
The International Center for Clinical Excellence (ICCE) is an
international online community designed to
support helping professionals, agency directors, researchers,
and policy makers improve the quality and outcome
of behavioral health service via the use of ongoing consumer
feedback and the best available scientific evidence.
The ICCE launched in December 2009 and is the fastest
growing online community dedicated to excellence in
clinical practice. Membership in ICCE is free. To join, go to:
www.centerforclinicalexcellence.com.
consist of a series of six guides covering the most
important information for practitioners and agencies
implementing FIT as part of routine care. The goal
for the series is to provide practitioners with a thorough
grounding in the knowledge and skills associated
with outstanding clinical performance, also known as the ICCE
Core Competencies. ICCE practitioners are
proficient in the following four areas:
Competency 1: Research Foundations
Competency 2: Implementation
Competency 3: Measurement and Reporting

Competency 4: Continuous Professional Improvement
2
The ICCE Manuals on FIT cover the following content areas:
Manual 1: What Works in Therapy: A Primer
Manual 2: Feedback-Informed Clinical Work: The Basics
Manual 3: Feedback-Informed Supervision
Manual 4: Documenting Change: A Primer on Measurement,
Analysis, and
Reporting
Manual 5: Feedback-Informed Clinical Work: Specific
Populations and
Service Settings
Manual 6: Implementing Feedback-Informed Work in Agencies
and Systems
of Care
Feedback-Informed Treatment (FIT) Defined
Feedback-Informed Treatment is a pantheoretical approach for
evaluating and improving the quality and
effectiveness of behavioral health services. It involves routinely

and formally soliciting feedback from consumers
regarding the therapeutic alliance and outcome of care and
using the resulting information to inform and tailor
service delivery. Feedback-Informed Treatment (FIT), as
described and detailed in the ICCE manuals, is not
only consistent with but also operationalizes the American
Psychological Association’s (APA) definition of
evidence-based practice. To wit, FIT involves “the integration
of the best available research…and monitoring
of patient progress (and of changes in the patient’s
circumstances – e.g., job loss, major illness) that may suggest
the need to adjust the treatment…(e.g., problems in the
therapeutic relationship or in the implementation of
the goals of the treatment)” (APA Task Force on Evidence-
Based Practice, 2006, pp. 273, 276-277).
3
In this manual, significant research findings that form the
foundation of Feedback-Informed Treatment (FIT) are
reviewed and discussed. Also included in this manual are a
short quiz, Frequently Asked Questions (FAQ), and a list of
references for the sources cited.
Significant Research Findings
In this section we review the major research findings that
provide empirical support for FIT. These findings are divided
into four parts:

1) Behavioral Health Outcomes;
2) The Therapeutic Alliance;
3) Properties of Alliance and Outcome
Measures; and
4) Expert Performance and Clinical
Practice.
What Works in Therapy:
A Primer
Manual 1
4
1) Behavioral Health Outcomes
Psychotherapy is an efficacious approach for the amelioration
of psychological distress and improvement of functioning. In
a major review of the available evidence, outcome researchers
Lambert and Ogles (2004) conclude that, “psychotherapy
facilitates the remission of symptoms and improves functioning.
It not only speeds up the natural healing process but also often
provides additional coping strategies and methods for dealing
with future problems. Providers as well as patients can be
assured
that a broad range of therapies, when offered by skillful, wise
and stable therapists, are likely to result in appreciable gains for

the client” (p. 180).
5
The average treated person
is better off than 80% of
those without the benefit of
treatment.
Research over the last 50 years has consistently
demonstrated that psychotherapy across
populations, age, gender, and diagnosis has a large
and consistent effect size, between 0.8 and 1.2
(Asay & Lambert, 1999; Lambert & Ogles, 2004;
Smith & Glass 1977; Smith, Glass, & Miller,
1980; Wampold, 2001). Effect size refers to the
magnitude of change (in standard deviation units)
attributable to treatment, either by comparing the
clinical status of a treated sample with that of an
untreated sample, or by comparing a sample’s
clinical status before and after treatment.
Psychotherapy is cost-
effective.
Therapy has been shown to reduce inpatient
stays, consultations with primary-care physicians,
use of medications, care provided by relatives,
and general health care expenditures by 60% to
90% (Chiles, Lambert, & Hatch, 1999; Kraft,

Puschner, Lambert, & Kordy, 2006). These
findings have been demonstrated with persons
with high-utilization rates of medical and health-
related services who received individual, family,
and marital therapy (Cummings, 2007; Law,
Crane & Berge, 2003).
Therapy works largely because
of general factors that
are expressed in variable
proportions through the
interactions between clinicians
and consumers.
Despite attempts to identify specific ingredients
in psychotherapy, research has found that a core
group of general therapeutic factors is responsible
for successful outcomes, regardless of the approach
or model (see Hubble, Duncan, & Miller, 1999;
Lambert, 1992; Lambert, Shapiro, & Bergin,
1986). In this manual the term therapeutic
factors is used to describe any factor known to
contribute to effective psychotherapy. As these
are presented and discussed, it is important to
remember that these factors are not invariant,
proportionally fixed, or neatly additive; they
are fluid and dynamic. The role and degree of
influence that any one factor has on outcome
is dependent on the context; specifically, who
is involved, what takes place between therapist
and client, when and where the therapeutic
interaction occurs and, ultimately, from whose
point of view these matters are considered. Figure
1 provides a visual representation of therapeutic
factors that research has shown account for the
outcome of psychological treatments.

Significant research findings in behavioral health outcomes
include:
6
Alliance Effects (5-8%)
Model/Technique
Effects (1%)
Expectancy, Placebo
and Allegiance Effects
(4%)
Therapist Effects
(4-9%)
Treatment Effects
Client/
Extratherapeutic
Factors
The Therapeutic factors
Therapeutic factors include and
are defined as:

Client/Extratherapeutic Factors.
These factors are independent of treatment and include
clients’ readiness for change, strengths, resources,
level of functioning before treatment (premorbid
functioning), social support systems, socioeconomic
status, personal motivations, and life events (Hubble
et al., 2010). It is estimated (e.g., Wampold, 2001)
that client/extratherapeutic factors account for 80-
87% of the variability in scores between treated
and untreated clients. Much of the variability is
attributable to general statistical error – unexplained,
uncontrolled, or unrecognized influences, including
the shortcomings and inevitable fallibility of
experimental methodology and measurement. Some
of those unexplained or unrecognized influences are
what clients bring to therapy, their circumstances
and events that take place in their lives while they
happen to be in therapy that either aid or hinder
improvement.
Treatment Effects. These effects
represent a broad class of factors that are considered
relevant to the influence of treatment. It is estimated
that treatment in total contributes about 13-20%
to overall outcome. Treatment effects include:
Figure 1

7
Therapist Effects: Research shows
that “who” provides the therapy is an important
determinant of outcome. Numerous studies
demonstrate that some clinicians are more
effective than others (e.g., Brown, Lambert,
Jones, & Minami, 2005; Luborsky et al., 1986;
Wampold & Brown, 2005). “Better” therapists,
it turns out, form better therapeutic relationships
with a broader range of clients. In fact, 97% of
the difference in outcome between therapists
is accounted for by differences in forming
therapeutic relationships (Baldwin, Wampold, &
Imel, 2007). By contrast, other therapist qualities
have little or no impact on outcome, including:
age, gender, years of experience, professional
discipline, degree, training, licensure, theoretical
orientation, amount of supervision or personal
therapy, and use of evidence-based methods.
Alliance Effects: The amount of change
attributable to the quality of the relationship
between therapist and client is due to alliance
effects. It turns out that the therapeutic relationship
is the largest contributor to outcome in behavioral
health services. In essence, the alliance works
by engaging the client in the treatment process.
Research shows that client level of engagement is
the most potent predictor of change in therapy.
E x p e c t a n c y , P l a c e b o , a n d
Allegiance Effects: These factors relate
to both the client and therapist’s expectations and

beliefs about therapy and its potential effects. For
the client, these effects relate to the installation
of hope and expectations about the healing
properties of therapy, and more specifically to the
client’s belief in the therapist and the treatment
provided (also known as the “placebo effect”).
For the therapist, these factors include positive
expectations, faith in therapy as a practice, and a
belief in (allegiance to) the approach and methods
utilized.
M o d e l / T e c h n i q u e E f f e c t s : All
therapies involve methods – healing rituals – the
effect of which depends on the degree to which
these methods fit with clients’ preferences and
expectations and activate other factors such as
placebo and hope to foster improvement. Models
and techniques work best when they engage
and inspire participants; and they can provide
structure to therapy. Studies have indicated that a
lack of structure and focus in treatment are good
predictors of a negative psychotherapy outcome
(e.g., Lambert & Bergin, 1994; Mohl, 1995;
Sachs, 1983).
Available evidence indicates that psychotherapy contributes to
symptom amelioration
and improved client functioning. It also is cost effective. With
the effectiveness and
benefit of therapy well-established, we move to address two
critical questions in the
sections that follow: “What does not work in therapy?” and
more positively, “What
does work in therapy?”

8
This section focuses on what does not work in
therapy by addressing two primary areas: (1) the lack
of overall improvement in therapy outcomes dating
back to the first meta-analytic studies in the 1970s;
and (2) a list of non-predictors and weak predictors
of outcome.
The Lack of Improvement in the
Effectiveness of Therapy
Available research points to the reasons why the
effectiveness of psychological treatments has not
improved appreciably over the last three decades.
• The emphasis on treatment models
in professional discourse and
training:
Though popular, there are actually few if any
meaningful differences in outcomes between
competing approaches – especially when the
following factors are taken into account:
Equal comparison conditions between bona fide
approaches intended to be therapeutic. Bona fide
approaches are defined as treatments that are: (1)

intended to be therapeutic (having a theoretical
base and associated techniques); (2) considered
viable by the psychotherapeutic community
(e.g., through professional books or manuals);
(3) delivered by trained therapists; and (4)
containing ingredients common to all legitimate
psychotherapies (e.g., a therapeutic relationship)
(Anderson, Lunnen, & Ogles, 2010; Benish et al.,
2008; Frank & Frank, 1991; Imel & Wampold,
2008; Wampold, 2007; Wampold et al., 1997). In
sum, when treatment conditions are equal there
are no discernible differences between bona fide
treatment approaches.
The statistical strength of meta-analytic studies
as compared to single studies. Meta-analyses are
a method of pooling together numerous studies
with varying methodologies, sample sizes, and
treatment approaches, all of which improves
statistical power, flexibility, and generalizability
compared to single studies. Numerous meta-
analyses find no difference in effect between bona
fide treatment approaches. To date, no differences
in outcome have been found between different
treatment approaches for psychotherapy in general
What Does Not Work in Therapy
9

(Wampold et al., 1997), depression (Wampold et
al., 2002), PTSD (Benish, Imel, & Wampold,
2008), alcohol use disorders, (Imel et al., 2008) and
the four most common diagnoses in children and
youth (depression, ADHD, anxiety, and conduct
disorder; Miller, Wampold, & Varhely, 2008).
Despite claims that certain methods are superior
to others, or that evidence-based practice is defined
by specific treatments for specific diagnoses,
meta-analytic studies fail to support such claims.
Furthermore, any differences between approaches
reported in specific studies do not exceed what
would be expected by chance (Wampold, 2001).
The failure to find any difference in effect between
competing treatment is referred to as “The Dodo
Verdict,” an expression first coined by psychologist
Saul Rozenzweig who borrowed a line of text from
Alice’s Adventures in Wonderland to summarize the
evidence regarding differential efficacy: “All have
won, and therefore all deserve prizes.”
• The failure to address dropouts
in psychotherapy:
Research to date suggests that premature
termination or dropout – the unilateral decision
by clients to end therapy – averages about 47%
(Wierzbicki & Pekarik, 1993). For children and
adolescents, the range varies from 28% to 85%
(Garcia & Weisz, 2002; Kazdin, 1996). Clinicians,
it turns out, achieve solid outcomes with clients
who stay but too many decide early to discontinue
services.

• The failure to identify which
consumers of behavioral health
services will not benefit and
which will deteriorate while in
care:
Even with well-trained and supervised clinicians,
a significant percentage (30% to 50%) of clients
do not benefit from therapy. Deterioration rates
among adult clients range between 5% and 10%
(Hansen, Lambert, & Forman, 2002; Lambert &
Ogles, 2004). Regarding children and adolescents,
rates of deterioration vary between 12% and 20%
(Warren et al., 2010). It is estimated that the
clients who do not benefit or deteriorate while in
psychotherapy are responsible for 60-70% of the
total expenditures in the health care system (Miller,
2011). Moreover, clinicians routinely fail to identify
clients who are not progressing, deteriorating, and
at most risk of dropout and negative outcome
(Hannan et al., 2005). Conversely, clinicians who
have access to outcomes data can better identify
clients who are not improving or getting worse
and respond to those clients, thereby reducing the
risk of dropout and negative outcome (Lambert,
2010; Miller et al., 2004).

10
• The substantial variation in
outcomes between clinicians
with similar training and
experience:
In practice settings, some psychotherapists
consistently achieve better outcomes than
others, regardless of the psychiatric diagnoses,
age, developmental stage, medication status,
or severity of the people they work with
across a range of patients (Brown, Lambert,
Jones, & Minami, 2005; Wampold & Brown,
2005). Findings indicate that clients of the
most effective therapists improve at a rate
at least 50% higher and drop out at a rate
at least 50% lower than clients who work
with less effective therapists (Wampold &
Brown, 2005). The latest research indicates
that 97% of the difference in outcome
between therapists is attributable to
differences in their ability to form alliances
with clients (Baldwin, Wampold, & Imel,
2007). Such findings indicate that the most
effective therapists work harder than their
counterparts at seeking and maintaining
client engagement, as well as invest more
time, energy, and resources into improving
their craft (Hubble et al., 2010). Research
consistently shows that the best predictor
of engagement in psychological services is

the client’s rating of the therapeutic alliance
(Bachelor & Horvath, 1999).
• Therapists’ lack of knowledge
regarding their overall rate of
effectiveness and the tendency
of average clinicians to
overestimate:
The majority of therapists have never measured
and do not know how effective they are (Hansen,
Lambert & Forman, 2002; Sapyta, Riemer, &
Bickman, 2005). Naturally, it is impossible for
clinicians to know if they are improving if they do
not know their level of effectiveness. Additionally,
therapists are not immune to a self-assessment
bias in terms of comparing their own skills with
those of their colleagues and in estimating the
improvement or deterioration rates likely to occur
with their clients (Dew & Reimer, 2003; Lambert,
2010). Walfish, McAlister, O’Donnell, and
Lambert (2010) found that therapists on average
rated their overall clinical skills and effectiveness
at the 80th percentile – a statistical impossibility.
Even worse, less than 4% considered themselves
average and not a single person in the study
rated his or her performance below average. The
issue of therapists overestimating their personal
effectiveness puts clients at risk for higher rates of
dropout and negative outcome.

11
• Clinician effectiveness tends to
plateau over time in the absence
of concerted efforts to improve
it:
During their careers, clinicians acclimate to
their settings, rely more on specific methods
and strategies with which they are trained or are
more comfortable, and become more confident in
what they believe to be true about their clientele.
Although these and other clinician factors may
benefit specific clients in specific situations, they
more often contribute to a plateauing of clinician
effectiveness. Clinicians need to establish personal
baselines of effectiveness and employ reliable
and valid methods to monitor and track client
feedback in relation to outcomes and the alliance
to improve on those baselines.
Non-predictors and weak/absent
predictors of outcome
Myriad studies over the last three decades have
identified variables that have little or no correlation
with the outcome of treatment, including:

• Consumer age, gender, diagnosis,
and previous treatment history
(Wampold & Brown, 2005)
• Clinician age, gender, years
of experience, professional
discipline, degree, training,
licensure, theoretical
orientation, amount of
supervision, personal therapy,
specific or general competence,
and use of evidence-based
practices (Beutler et al., 2004; Hubble et al.,
2010; Nyman, Nafziger, & Smith, 2010; Miller,
Hubble, & Duncan, 2007; Wampold & Brown,
2005)
• Model/technique of therapy
(Benish, Imel, & Wampold, 2008; Imel et
al., 2008; Miller, Wampold, & Varhely, 2008;
Wampold et al., 1997; Wampold et al., 2002)
• Matching therapy to diagnosis
(Wampold, 2001)
• Adherence/fidelity/competence to
a particular treatment approach
(Duncan & Miller, 2005; Webb, DeRubeis, &
Barber, 2010)

12
What Does Work in Therapy
This section focuses on two areas that form the
foundation of what works in therapy: (1) evidence
of the role of routine and ongoing client feedback in
improving outcomes; and (2) predictors of outcome.
Each area is central not only to improving outcomes,
but also in elevating consumer confidence, ensuring
the long-term viability of psychotherapy as a
treatment option and creating greater accountability,
stewardship and return on mental health service
investments. There is a worldwide shift toward
outcomes that is not specific to mental health. It is
essential that clinicians follow
this lead and demonstrate
– through reliable and valid
methods – a greater degree of
accountability for the value of
psychotherapy.
Evidence of the
Role of Routine and
Ongoing Feedback in
Improving Outcomes
The best available research
reveals that the use of routine
and ongoing client feedback

provides practitioners and the
field with a simple, practical,
and meaningful method for
documenting the usefulness of treatment. Seeking
and obtaining valid, reliable, and feasible feedback
from consumers regarding the therapeutic alliance
and outcome as much as doubles the effect size of
treatment, cuts dropout rates in half, and decreases
the risk of deterioration. As the APA Task Force
on Evidence-Based Practice (2006) concludes,
“providing clinicians with real-time patient feedback
to benchmark progress in treatment and clinical
support tools to adjust treatment as needed” is one
of the “most pressing research needs” (p. 278).
Miller (2011) summarized the impact of routinely monitoring
and
using outcome and alliance data from 13 RCTs involving 12,374
clinically, culturally, and economically diverse consumers and
found:
• Routine outcome monitoring and feedback as
much as doubles the
“effect size”(reliable and clinically significant
change);
• Decreases dropout rates by as much as
half;
• Decreases deterioration by 33%;
• Reduces hospitalizations and shortens length of
stay by 66%;
• Significantly reduces cost of care compared to

non-feedback groups
(which increased in cost).
Additional evidence indicates that regular, session-by-session
feedback (as opposed to less frequent intervals, i.e., every third
session, pre- and post-services, etc.; Warren et al., 2010) is
more
effective in improving outcome and reducing dropouts.
13
Predictors of outcome
The following factors have been shown to be
consistent, robust predictors of eventual outcome.
• Duration of therapy without
positive change:
The longer clients attend therapy without
experiencing a positive change, the greater the
likelihood they will experience a negative or null
outcome or drop out (Duncan, Miller, Wampold
& Hubble, 2010).
• Early client change:
Referred to as the “dose-effect relationship”
in psychotherapy, research indicates that

approximately 30% of clients improve by the
second session, 60% to 65% by session seven,
70% to 75% by six months, and 85% by one
year (Howard, Kopta, Krause, & Orlinksy, 1986).
Although the rate of client change differs somewhat
from person to person, early response in therapy is
a strong indicator of eventual outcome, making
the monitoring of improvement from the start of
therapy essential.
• Consumer rating of the alliance:
The client’s rating of the alliance is a consistent
and reliable predictor of treatment outcome. A
significant body of evidence further indicates that
the client’s rating is superior to the therapist’s as a
predictor of retention in treatment and eventual
outcome.
• Level of consumer engagement:
Orlinsky, Rønnestad, and Willutzki (2004)
observe, “the quality of the patient’s participation…
[emerges] as the most important determinant in
outcome” (p. 324). Clients who are more engaged
and involved in therapeutic processes are likely
to receive greater benefit from therapy. The best
predictor of client engagement is the alliance. In
this regard, recall findings cited earlier showing
that most of the difference in outcome between
clinicians was in the ability to form, nurture, and
sustain alliances with diverse clients.
• Improvement in the alliance over

the course of treatment:
Client-therapist alliances that strengthen and
improve from intake to termination tend to yield
better outcomes than alliances which start and stay
good or deteriorate over time (Anker, Duncan, &
Sparks, 2009; Anker et al., 2010). (See also Section
2.)
14
• The client’s level of distress at
the start of therapy:
More so than diagnosis, the severity of the client’s
distress at intake predicts eventual outcome.
Clients with higher levels of distress are more
likely to show measured benefit from treatment
than those with lower levels or those who present
as non-distressed (Duncan, Miller, Wampold &
Hubble, 2010). Knowledge about client distress
can inform decisions regarding the dose and
intensity of services.
• Clinician allegiance to their
choice of treatment approach:
While research shows few if any meaningful

differences in outcome among treatment
approaches, research documents that clinicians
must have faith in the restorative power of therapy
as a healing ritual. Further, it is important that
clinicians have therapeutic rationales, employ
strategies consistent with those rationales, and
believe in their approaches (Hubble et al., 2010).
15
The therapeutic alliance refers to the quality and strength of
the collaborative relationship between the client and therapist
(Norcross, 2010). The alliance is comprised of four empirically
established components: (1) agreement on the goals, meaning
,or purpose of the treatment; (2) agreement on the means and
methods used; (3) agreement on the therapist’s role (including
being perceived as warm, empathic, and genuine; and (4)
accommodating the client’s preferences. Over 1,100 separate
research findings document the importance of the alliance in
successful therapy, making it one of the most evidence-based
concepts in the psychotherapy (Norcross, 2011; Orlinsky,
Rønnestad, & Willutzki, 2004). Significant findings from this
research are detailed in this section.
2) The Therapeutic Alliance

16
The therapeutic alliance makes
substantial and consistent
contributions to client success
across different types of
psychotherapy.
Over 20 meta-analyses have demonstrated the
impact of the therapeutic alliance on treatment
outcome (Norcross, 2011). The relationship and
alliance act in concert with treatment methods, client
characteristics, and clinician qualities in determining
effectiveness. The alliance accounts for between five
to nine times more of the outcome of treatment than
the model or technique.
Next to the level of consumer
functioning at intake, the
consumer’s rating of the
alliance is the best predictor of
treatment outcome and is more
highly correlated with outcome
than clinician ratings.
The partnership between the therapist and client,
as rated by the client, is a consistent predictor of
eventual treatment outcome and more reliable
than therapist ratings (Horvath & Symonds, 1991;
Martin, Garske, & Davis, 2000; Norcross, 2010,
2011). Some therapists form better alliances with
clients and achieve better outcomes. In contrast,
clients of therapists with weaker alliances tend to drop
out at higher rates and experience poorer outcomes

(Hubble et al., 2010; Lambert, 2010).
A significant portion of the
variability in outcome between
clinicians is due to differences in
the therapeutic alliance.
Variability between clients is to be expected with
regard to client ratings of the alliance. However,
some therapists consistently form better alliances
with clients and variability in the alliance accounts
for a large portion of the differences in outcomes
between therapists (Baldwin et al., 2007).
Monitoring the alliance allows
clinicians to identify and correct
problems with engagement and
reduce early dropout or risk of
negative outcome.
Routine and ongoing monitoring of the alliance
through real-time client feedback processes helps
to both identify potential ruptures and create
opportunities for clinicians to take corrective steps
(Anker, Duncan, & Sparks, 2009; Anker et al., 2010).
In addition, improvements in the alliance (intake to
termination) are associated with better outcomes and
lower dropout rates (Duncan, Miller, Wampold, &
Hubble, 2010; Harmon et al., 2007; Lambert, 2010;
Miller, Hubble, & Duncan, 2007).

17
Two key factors have proven useful in predicting and improving
treatment outcome: (1) the quality of the alliance; and (2)
early change in treatment. Literally hundreds and hundreds
of outcome and alliance measures exist. Few, however, have
documented validity and reliability. Fewer still are feasible for
use in routine clinical care and are sensitive to change. In the
material that follows, properties of valid, reliable, and feasible
alliance and outcome measures that are sensitive to change
are defined and illustrated. Thereafter, the properties of the
outcome and alliance scales used in FIT will be reviewed.
3) Properties of Alliance and Outcome Measures
18
Reliability:
To be reliable, any differences between two
administrations of the same measurement tool
must be attributable to the changes in the variable
being measured. A thermometer, for example, that
gives different readings under similar circumstances
is not trustworthy and would not be deemed a
reliable measure of the temperature. Similarly, scores
that vary in spite of little or no difference in the

functioning of or relationship with the client would
not be considered reliable measures of outcome or
alliance.
Validity:
To be valid, evidence must be provided to show
that a scale measures what it purports to measure.
With outcome and alliance scales, this is most often
accomplished by correlating a scale with other
well-established or documented scales (known as
concurrent validity), testing whether the measure
can accurately differentiate between clinical groups
and non-clinical groups (discriminate validity),
reviewing and estimating whether the scale measures
what it purports to measure (face validity).
Feasibility:
Feasibility is the degree to which an instrument can
be explained, completed, and interpreted quickly and
easily. Available evidence indicates that any measure
or combination of outcome and alliance measures
taking more than five minutes to complete, score, and
interpret are less likely to be used by clinicians and
increase the likelihood of complaints by consumers
of mental health services (Duncan, Miller, & Sparks,
2004). As a result, a strong argument can be made
that, in addition to being reliable and valid, any
outcome and alliance tool employed in routine
clinical care must also be brief. Importantly, research
provides little evidence that longer and less feasible
(multifactor) outcome and alliance measures are
more useful in predicting, evaluating, and guiding
treatment than shorter (single-factor, general distress)

scales.
Sensitivity to Change:
To be useful for evaluating the practice of
psychotherapy, outcome and alliance scales need to
be sensitive to change among those receiving services
but return stable (or unchanging) scores among
those who do not receive treatment. An instrument’s
sensitivity to change enables researchers, clinicians,
and clients to be confident that any resulting changes
are attributable to the services being offered.
The Outcome and Session Rating
Scales:
The ORS and SRS are the measures of outcome
19
and alliance used in feedback-informed treatment.
Multiple studies have proven the measures to be
valid, reliable, feasible, and sensitive to change
(Miller, 2011; Duncan, Miller, Wampold, & Hubble,
2010). Both instruments take less than a minute to
administer, score, and interpret (examination copies
are available in Manual 2 of this series). As noted
earlier, studies conducted to date document that
routine use of the ORS and SRS in clinical practice
improves outcome, cuts dropout rates, and decreases

the cost of and time spent in treatment (Miller,
2011). Detailed instructions for using the measures
to inform and improve behavioral health service
delivery can be found in Manual 2.
20
A significant body of research across multiple
domains (e.g., medicine, music, sports, mathematics)
documents the steps required for achieving superior
performance. The steps are: (1) establishing a
performance baseline; (2) engaging in deliberate,
reflective practice; and (3) obtaining ongoing
feedback/coaching. These steps are reviewed below.
Establishing a baseline
performance level:
Whether in sports, music, medicine, or
psychotherapy, top performers are able to accurately
assess their knowledge, skills, and effectiveness.
What’s more, the best are always comparing their
current performance to: (1) their own personal best;
(2) the performance of others; and (3) a known
national standard or benchmark (Ericsson, Charness,
Feltovich, & Hoffman, 2006).
Engaging in deliberate, reflective
practice:

Expert performers engage in a specific form of practice
designed to improve individual target performance
4) Expert Performance and Clinical Practice
just beyond their current level of proficiency. The best
engage in such efforts up to four hours a day, every
day of the week, including weekends and holidays.
This highly focused, deliberate effort is extremely
taxing. As a result, most practice periods last no
longer than 45 minutes at a time and are followed by
periods of rest.
What constitutes deliberate practice differs across
domains of expertise. All forms, however, include the
highly focused, repetitious practice of skills focused
on improving the parts of performance that are not
yet mastered.
Over time, deliberate practice results in the
development of what researchers refer to as, “deep
domain-specific knowledge.” The best not only know
more, but also when, where, how, and with whom to
use what they know.
Obtaining ongoing feedback/
coaching:
Expert performers are usually guided in their practice
by a coach or mentor who provides directions for
practice that will push professionals just beyond their
current realm of reliable performance.

21
Research on expertise makes clear that, in order to
improve, clinicians need to: (1) measure outcomes
and determine their overall rate of effectiveness; (2)
identify areas of practice just beyond their current
level of proficiency; (3) develop and execute a
plan of deliberate practice; (4) obtain coaching,
instruction and/or training; (5) measure the
impact of the plan and training on performance;
and (6) adjust the plan and steps.
The above noted process has been termed TAR:
Think, Act, and Reflect. To move beyond the
realm of reliable performance, the best engage
in forethought. This means setting specific goals
for improvement and developing a plan to reach
those goals. In the act phase, successful experts
track their performance: they monitor on an
ongoing basis whether they used each of the steps
or strategies outlined in the thinking phase and
the quality with which each step was executed.
The sheer volume of detail gathered in assessing
their performance distinguishes the exceptional
from their more average counterparts. During
the reflection phase, top performers review the
details of their performance, identifying specific
actions and alternative strategies for reaching their
goals. Where unsuccessful learners paint in broad
strokes, attributing failure to external factors and
uncontrollable events, the best know exactly what
they do, most often citing controllable factors.

The findings from the expert performance are
directly applicable to mastering the knowledge and
skills associated with feedback-informed practice:
• Accept that mastering FIT will take time;
• Schedule time each day to study and
practice,
spending no more than 45 minutes at a time,with
periods of rest in between (15 minutes minimum);
• Discuss FIT with a more knowledgeable peer or
colleague (joining the ICCE provides instant
access);
• Set small, measurable goals and identify
discrete
indicators of performance (a good place to start is
completing the quiz that follows).
Implications for Therapists
22
Manual 1 Quiz
Research indicates that people retain knowledge better when
tested. Take a few moments and answer the

following 10 questions. If you miss more than a couple, go back
and reread the applicable sections. One week
from now, complete the quiz again as a way of reviewing and
refreshing what you have learned.
1. The best therapy outcomes are likely when the
following pattern of alliance scores are found:
a. Start good, end good.
b. Start good, end fair.
c. Start fair, end good.
d. Start poor, end good.
2. The factors that contribute most to therapeutic
change, going from least to most, are:
a. Technique, theory, alliance, placebo.
b. Theory and technique, allegiance, alliance.
c. Therapist, theory, technique, alliance.
d. Theory, allegiance, alliance, diagnosis.
3. Why is it recommended to measure the
alliance and outcome with clients at every
visit?
a. To optimize opportunities to adjust and
improve treatment.
b. To provide supervisors earlier opportunities
to correct therapists.

c. To encourage earlier termination.
d. To make the administration of the
measure more automatic and less prone to
discussion.
4. Which of the following statements is true?
a. There is ample evidence to prove that some
therapeutic approaches are more effective
than others for treating certain disorders.
b. All treatment approaches work about
equally well.
c. Technique makes the largest percentage-wise
contribution to treatment outcome.
d. Dismantling studies show that certain
specific ingredients are necessary for
therapeutic effectiveness.
5. The therapeutic alliance is made up of the
following components:
a. Positive outcome, agreement on methods,
consumer preferences, bond between
consumer and provider.
b. Therapist empathy, level of engagement,
agreement on goals, agreement on methods.
c. Consumer preferences, agreement on goals,
agreement on methods, bond between
consumer and provider.

d. Consumer preferences, client strengths,
client compliance to treatment, client’s
belief in treatment.
23
6. It is a good idea to monitor the alliance on an
ongoing basis because:
a. It allows clinicians to identify and reduce
the risk of early dropout or null or negative
change.
b. It allows clients to be more assertive.
c. It helps the clinician feel supported by the
large proportion of high scores.
d. It allows agencies to identify the least
effective therapists based on the low alliance
scores.
7. What is the average deterioration rate in
psychological treatments?
a. Between 20-25% of clients deteriorate while
in treatment.
b. Between 5-10% of clients deteriorate while

in treatment.
c. Between 0-5 % of clients deteriorate while
in treatment.
d. Between 30-40% of clients deteriorate while
in treatment.
8. Which of the following is a predictor of
outcome?
a. Consumer diagnosis.
b. Early positive change.
c. Clinician licensure, discipline, training,
degrees, personal therapy, certifications,
clinical supervision.
d. Consumer’s previous treatment history.
9. Why is it recommended to measure outcome
and alliance of your clients at every session?
a. Measuring outcomes and alliance in real-
time provides the treatment provider an
opportunity to adjust the treatment in order
to maximize the potential for a positive
therapeutic outcome from the client’s
perspective.
b. This allows your supervisors and managers
to know if you are doing your job.
c. This will convince your client that you will
definitely be helpful to him or her.

d. None of the above.
10. Which of the following statements is false?
a. Improvement as a clinician is directly related
to how much continuing education one has
completed.
b. The first step in improving as a clinician is
to be able to calculate one’s own baseline of
performance.
c. Both a and b are true.
d. Both a and b are false.
1. d
2. b
3. a
4. b
5. c
6. a
7. b
8. b
9. a
10. a

Answer Key
24
FAQ
Question
If all models are equally effective, does that mean that you can
do whatever you want – that having
a technique doesn’t matter?
Answer
No, it doesn’t mean that having a theory and technique doesn’t
matter. Theory and technique
achieve their effects through the activation and operation of
placebo, hope and expectancy in the
client. Having a theory and technique provides the therapist
with a set of beliefs and procedures
unique to the specific approaches, a rationale for the client’s
difficulties, strategies or “healing
rituals” to follow for problem resolution. The theory and
technique provide a structure and focus
without which can result in a disorganized or “hit and miss”
approach.
Question

How come some meta-analyses show there is a difference
between the effectiveness of models but
this manual says that there is no difference?
Answer
The main explanation is that the results of a meta-analysis
depend on the studies included. Some
meta-analyses include studies that are not direct comparisons
between bona-fide treatments,
leading to the mistaken conclusion that some treatments are
more effective than others. Such
studies don’t control for researcher allegiance effects (e.g., a
researcher’s belief in the superiority of
his or her chosen model) and include “unfair comparisons”
(e.g., where treatment is compared to
wait list conditions, to psychoeducation or to other types of
interventions that are not equivalent
with the treatment being offered). In short, the validity of a
meta-analysis depends on the studies
reviewed. (For a more detailed description, read Imel et al.,
2008.)
25
Question
How come this manual states that a professional degree,
training, and years of experience have no
impact on client outcome? Does this mean my professional

training is not necessary? How am I
supposed to improve my skills if what I have done so far is not
effective?
Answer
The main conclusion from research on the development of
expertise is improvement results from
practicing in a very specific, deliberate, and focused way.
Experience is not enough. Daily work
with clients will not improve outcome if such “deliberate
practice” is absent from in the work. In
fact, therapeutic skills and outcomes plateau and even
deteriorate over time due to the absence
of focused practice of the basic therapeutic skills related to the
specific context of therapy. More
detail on this subject can be found in the “Expert Performance
and Clinical Practice” section under
“Implications for Therapists.”
Question
Does the idea of “deliberate practice” mean that I just need to
see as many clients as possible each
week in order to become a superior therapist?
Answer
No. In fact, research shows no correlation between the number
of hours spent conducting
therapy and effectiveness. Improving one’s skills requires being
pushed beyond one’s current level
of proficiency. The process includes first identifying areas for
improvement, setting small goals,
developing and implementing an action plan, and then
reviewing the results and adjusting the

plan. Without deliberate effort, superior results remain elusive.
You can read more about this in the
“Expert Performance and Clinical Practice” section under
“Implications for Therapists.”
26
Question
This manual states that “Even with well-trained and supervised
clinicians, 30% to 50% of clients
do not benefit from therapy and fail to respond to treatment.”
Does this mean 30% to 50% of
clients will never improve?
Answer
No. Recall that successful treatment is about the FIT between a
particular client and therapist.
Perhaps the client needs something other than the intervention
being offered by a given therapist.
Perhaps problems in the alliance stand in the way of success.
The good news is that once a client is
referred to another clinician or program, available evidence
indicates that the probability of success
is unaffected by the prior treatment failure.
Question
This manual states that “early change is predictive of outcome

at the end of treatment.” I have
heard from colleagues and supervisors that clients need to get
worse in order to get better. Is this
not true?
Answer
There is no empirical evidence supporting the statement that
“clients need to get worse in order
to get better.” Some clients do deteriorate. Understandably,
however, they are at increased risk for
dropping out of service. This is a theoretical assumption that
might fit with some but not across all
clients. Early change is the pattern supported by research.
Question
How do I start getting feedback? Which tool should I use and
where can I find it?
Answer
You can download the Outcome Rating Scale (ORS) and Session
Rating Scale (SRS) for free at
www.centerforclinicalexcellence.com. These are valid,
reliable,and feasible tools with each taking
under a minute to administer and score. Manual 2 provides
detailed information regarding the
application of ORS and SRS in clinical practice.

27
Anderson, T., Lunnen, K. M., & Ogles, B. M. (2010).
Putting models and techniques in context. In B.
L. Duncan, S. D. Miller, B. E. Wampold, & M.
A Hubble (Eds.), The heartand soul of change:
Delivering what works in therapy (2nd ed.) (pp. 143-
166). Washington, DC: American Psychological
Association.
Anker, M. G., Duncan, B. L., & Owen, J., Sparks, J.
A. (2010). The alliance in couple therapy: Partner
influence, early change, and alliance patterns in a
naturalistic sample. Journal Consulting and Clinical
Psychology, 78(5), 635-645.
Anker, M. G., Duncan, B. L., & Sparks, J. A. (2009).
Using client feedback to improve couple therapy
outcomes: A randomized clinical trial in a naturalistic
setting. Journalof Consulting and Clinical Psychology,
77(4), 693-704.
APA Presidential Task Force on Evidence-Based Practice.
(2006). Evidence-based practice in psychology.
American Psychologist, 61(4), 271–285.
Asay, T. P., & Lambert, M. J. (1999). The empirical
case for the common factors in therapy: Quantitative
findings. In M. A. Hubble, B. L. Duncan, & S. D.
Miller (Eds.), The heartand soul of change: What
works
in therapy (pp. 33–56). Washington, DC: APA Press.
Bachelor, A., & Horvath, A. (1999). The therapeutic
relationship. In S. D. Miller (Ed.), The heart and
soul of change: What works in therapy (pp.

133–
Association.
Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007).
Untangling the alliance-outcome correlation:
Exploring the relative importance of therapist and
patient variability in the alliance. Journal of Consulting
and Clinical Psychology, 75(6), 842–852.
References
Benish, S., Imel, Z. E., & Wampold, B. E. (2008). The
relative efficacy of bona fide psychotherapies of post-
traumatic stress disorder: A meta-analysis of direct
comparisons. Clinical Psychology Review, 28, 746-758.
Beutler, L. E., Malik, M., Alimohamed, S., Harwood,
T. M., Talebi, H., & Noble, S. (2004). Therapist
variables. In M.J. Lambert (Ed.), Bergin and Garfield’s
handbook of psychotherapy and behavior change (5th
ed.)(pp. 227-306). New York: Wiley.
Brown, G.S., Lambert, M.J., Jones, E.R., & Minami, T.
(2005). Identifying highly effective psychotherapists
in a managed care environment. American Journal of
Managed Care, 11(8), 513-520.
Chiles, J., Lambert, M. J., & Hatch, A. L. (1999). The
impact of psychological interventions on medical cost
offset: A meta-analytic review. Clinical Psychology,
6(2), 204–220.
Cummings, N. A. (2007). Treatment and assessment
take place in an economic context, always. In S. O.

Lilienfeld & W. T. O’Donohue (Eds.), The greatideas
of clinical science: 17 principles that every mental
health
professional should understand (pp. 163–184). New
York: Routledge.
Dew, S., & Reimer, M. (2003). Why inaccurate
self-evaluation of performance justifies feedback
interventions. In L. Bickman (Chair), Improving
outcomes through feedback intervention. Symposium
conducted at the 16th Annual Research Conference,
A System of Care for Children’s Mental Health:
Expanding the Research Base, Tampa, University of
South Florida, The Louis de la Parte Florida Mental
Health Institute , Research and Training Center for
Children’s Mental Health.
Duncan, B. L., & Miller, S. D. (2005). Treatment
manuals do not improve outcomes. In J. C Norcross,
28
L. E. Beutler, & R. F. Levant (Eds.), Evidence-based
practices in mental health: Debate and dialogue on
the
fundamental questions (pp. 140-160). Washington,
DC: American Psychological Association.
Duncan, B.L., Miller, S.D., & Sparks, J. (2004). The
Heroic Client (2nd ed.). San Francisco, CA: Jossey-

Bass.
Duncan, B. L., Miller, S. D., Wampold, B. E., &
Hubble, M. A. (Eds.). (2010). The heart and soul
of change: Delivering what works in therapy
(2nd
ed.). Washington, DC: American Psychological
Association.
Ericsson, K. A., Charness, N., Feltovich, P., & Hoffman,
R. (Eds.). (2006). The Cambridge handbook of
expertise
and expert performance. New York: Cambridge
University Press.
Frank, J. D., & Frank, J. B. (1991). Persuasion and
healing: A comparative study of psychotherapy
(3rd ed.).
Baltimore: Johns Hopkins University Press.
Garcia, J. A., & Weisz, J. R. (2002). When youth
mental health care stops: Therapeutic relationships
problems and other reasons for ending youth
outpatient treatment. Journal of Consulting and
Clinical Psychology, 70(2), 439-443.
Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S.
L., Smart, D. W., Shimokawa, K., et al. (2005). A
lab test and algorithms for identifying clients at risk
for treatment failure. Journal of Clinical Psychology: In
Session, 61, 155-163.
Hansen, N., Lambert, M. J., & Forman, E. M. (2002).
The psychotherapy dose-response effect and its
implication for treatment delivery services. Clinical

Psychology: Science and Practice, 9(3), 329–343.
Harmon, C., Hawkins, E. J., Lambert, M. J., Slade, K.,
& Whipple, J. L. (2005). Improving outcomes for
poorly responding clients: The use of clinical support
tools and feedback to clients. Journal of Clinical
Psychology, 61(2), 175–185.
Horvath, A. O., & Symonds, B. D. (1991). Relation
between working alliance and outcome in
psychotherapy: A meta-analysis. Journal of Consulting
and Clinical Psychology, 38(2), 139–149.
Howard, K. I., Kopte, S. M., Krause, M. S., & Orlinsky,
D. E. (1986). The dose-effect relationship in
psychotherapy. American Psychologist, 41(2), 159–164.
Hubble, M. A., Duncan, B. L., Miller, S. D., &
Wampold, B. E. (2010). Introduction. In B. L.
Duncan, S. D. Miller, B. E. Wampold, & M.
A. Hubble (Eds.), The heartand soul of change:
Delivering what works in therapy (2nd ed.)(pp. 23-
Association.
Hubble, M. A., Duncan, B. L., & Miller, S. D. (Eds.).
(1999). The heartand soul of change: What works
in
therapy. Washington, DC: American Psychological
Association.
Imel, Z. E., & Wampold, B. E. (2008). The common
factors of psychotherapy. In S. D. Brown & R. W.
Lent (Eds.), Handbook of counseling psychology
(4th
ed.)(pp. 249-266). New York: Wiley.

Imel, Z. E., Wampold, B. E., Miller, S. D., & Fleming,
R. R. (2008). Distinctions without a difference:
Direct comparisons of psychotherapies for alcohol use
disorders. Journal of Addictive Behaviors, 22, 533-543.
Kazdin, A. E. (1996). Dropping out of child
psychotherapy: Issues for research and implications
for practice. Child Clinical Psychology, 1, 133-156.
29
Kraft, S., Puschner, B., Lambert, M. J., & Kordy, H.
(2006). Medical utilization and treatment outcome
in mid- and long-term outpatient psychotherapy.
Psychotherapy Research, 16(2), 241–249.
Lambert, M. J. (1992). Implications of outcome
research for psychotherapy integration. In J. C.
Norcross & M. R. Goldfried (Eds.), Handbook of
psychotherapy integration (pp. 94–129). New York:
Basic Books.
Lambert, M. J. (2010). Prevention of treatment failure:
The use of measuring, monitoring, and feedback
in clinical practice. Washington, DC: American
Psychological Association.
Lambert, M. J., & Bergin, A. E. (1994). The
effectiveness of psychotherapy. In A. E. Bergin & S.

L. Garfield (Eds.), Handbook of psychotherapy and
behavior change (4th ed.) (pp. 143–189). New York:
Wiley.
Lambert, M. J., & Ogles, B. M. (2004). The efficacy
and effectiveness of psychotherapy. In M. J. Lambert
(Ed.), Bergin and Garfield’s handbook of psychotherapy
and behavior change (5th ed.) (pp. 139–193). New
York: Wiley.
Lambert, M. J., Shapiro, D. A., & Bergin, A. E. (1986).
The effectiveness of psychotherapy. In S. L. Garfield
& A. E. Bergin (Eds.), Handbook of psychotherapy and
behavior change (3rd ed.) (pp. 157–211). New York:
Wiley.
Law, D. D., Crane, D. R., & Berge, D. M. (2003). The
influence of individual, marital, and family therapy
on high utilizers of health care. Journal of Marital and
Family Therapy, 29(3), 353–363.
Luborsky, L., Crits-Christoph, P., McLellan, A.T.,
Woody, G., Piper, W., Liberman, B., Imber, S., &
Pilkonis, P. (1986). Do therapists vary much in
their success? Findings from four outcome studies.
American Journal of Orthopsychiatry, 56(4), 501-512.
Martin, D. J., Garske, J. P., & Davis, M. K. (2000).
Relationship of the therapeutic alliance with outcome
and other variables: A meta-analytic review. Journal of
Consulting and Clinical Psychology, 68(3), 438–450.
Miller, S. D. (2011). Psychometrics of the ORS and
SRS. Results from RCTs and Meta-analyses of
Routine Outcome Monitoring & Feedback. The

Model of TreatmentEducation and its EvaluationProblem.docx

Model of TreatmentEducation and its EvaluationProblem.docx

Recommended

Recommended

More Related Content

Similar to Model of TreatmentEducation and its EvaluationProblem.docx

Similar to Model of TreatmentEducation and its EvaluationProblem.docx (15)

More from helzerpatrina

More from helzerpatrina (20)

Recently uploaded

Recently uploaded (20)

Model of TreatmentEducation and its EvaluationProblem.docx