This document summarizes a presentation about opening up clinical performance variation and financial incentives in primary care quality of care. It discusses the Quality and Outcomes Framework (QOF), a pay-for-performance program introduced in 2004 as part of a new GP contract in the UK. The QOF rewards general practices for achieving quality targets in chronic disease care. It has expanded over time to include more indicators and domains. While initially estimated to cost £1.8 billion over 3 years, the program has cost over £9 billion after its first 9 years. The presentation examines research on the impact and effectiveness of the QOF program.

1. Opening up clinical performance
variation and financial incentives in Primary Care quality of care
Evan (Evangelos) Kontopantelis1
1Institute of Population Health
Faculty Research Series, 25 September 2013
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

2. Outline
1 People
2 The scheme...
3 The research journey
4 Summary
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

3. Collaborative work!
Martin Roland
Tim Doran
David Reeves
Stephen Campbell
Bonnie Sibbald
Matt Sutton
Hugh Gravelle
Jose Valderas
...and others...
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

4. Improving quality of care
a (very) juicy carrot...
A pay-for-performance (p4p) program kicked off in April 2004 with
the introduction of a new GP contract
General practices are rewarded for achieving a set of quality targets
for patients with chronic conditions
The aim was to increase overall quality of care and to reduce
variation in quality between practices
The incentive scheme for payment of GPs was named the Quality
and Outcomes Framework (QOF)
Initial investment estimated at £1.8 bn for 3 years (increasing GP
income by up to 25%)
QOF is reviewed at least every two years
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

5. Quality and Outcomes Framework
details for years 1 (2004/5) and 7 (2010/11)
Domains and indicators in year 1 (year 7):
Clinical care for 10 (19) chronic diseases, with 76 (80) indicators
Organisation of care, with 56 (36) indicators
Additional services, with 10 (8) indicators
Patient experience, with 4 (5) indicators
Implemented simultaneously in all practices (a control group was
out of the question)
Practices are allowed to exclude patients from the indicators and
the payment calculations
Into the 10th year now (01Mar13/31Apr14); cost for the first 9
years was well above the estimate at £9 bn approximately
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

6. Quality and Outcomes Framework
what does it mean?
If you have
Atrial fibrillation, asthma, HT, cancer, CHD,
HF, CKD, COPD, dementia, depression,
diabetes, epilepsy, SMI, osteoporosis, PAD,
stroke, hypothyroidism
but also covers LD, obesity, palliative care,
sexual health, smoking
e.g. for diabetes
measure-control BP, chol, glucose
immunise for influenza
physical exams (retinal screen, foot exam)
...and more... 17 indicators in total
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

7. Quality and Outcomes Framework
2005 reaction
Drugs cut strokes by a third
Doctors get 20%
pay rise just for
doing their jobsDOCTORS are getting 20 per
cent pay rises for doing simple
tasks “they should always have
done,” it was claimed last night.
Health economists, including
Government advisers, say that
under their new contracts GPs
are being paid huge bonuses to
meet “easy” targets that do little
to improve patients’ lives.
Such is the level of concern
over the value for taxpayers that
ministers are already meeting
doctors’ leaders in order to set
tougher goals.
The move comes just weeks
after the Government boasted
how most GPs had met the tar-
gets for performance-related pay.
But Alan Maynard, professor of
health economics at York Univer-
sity, said yesterday: “It’s quite
ridiculous. The Government’s
CHOLESTEROL -lowering
drugs called statins could pre-
vent heart attacks and strokes in
a third of patients with diseased
arteries, a study shows.
Most doctors consider statin
treatment only when blood cho-
lesterol is above a certain level
but research by British and
Australian scientists shows
many more people with lower
cholesterol could also benefit.
Statins work by blocking the
action of an enzyme that enables
the liver to produce cholesterol.
Research Council scientist Dr
Colin Baigent, who coordinated
By Michael Day
By Geoff Marsh
spent all this money and has
given GPs 20 per cent rises just to
try and get them to do what they
should always have done.
“Now they’re being paid extra
to look out for people with high
blood pressure, which causes
heart attacks and strokes, and to
monitor and treat them – but they
should have been doing that all
along. It’s not rocket science.”
There have already been two
Whitehall meetings after the dis-
closure last month that, on aver-
age, GP practices achieved 91 per
cent of the available perfor-
mance-related bonuses under the
new contract.
After achieving bonuses, aver-
age earnings of a practice surged
by £75,000 while the salary of the
average GP partner rose to
£100,000 a year. The first meeting
to set tougher targets came with-
in a week of the announcement.
The second was held last week.
One BMA negotiator, Leeds GP
Richard Vautrey, said: “We are
looking at making amendments,
but at present we’re not in
the position to say exactly what
these are.
“We would expect the revised
framework to be ready by the end
of the year.”
Apart from blood pressure,
other target areas have been sin-
gled out as too modest. Chris
Ham, professor of health policy
and management at Birmingham
University, said: “My view is that,
in principle, the new GP contract
is a good thing – GPs are being
paid on the basis of how well they
treat patients and not just accord-
ing to how large their list size is.
“But the fact so many of them
met the targets in the first year
suggests they were too easy.”
A BMA spokeswoman denied
the targets were too easy. “They
demonstrate the vast majority of
doctors are already providing
high-quality care for their
patients,” she said.
Simon William, director of pol-
icy at the Patients’ Association,
said, however: “The BMA would
trumpet the GPs’ performance.
It’s a trade union, it’s there to
represent doctors not patients.”
A spokesman for the Depart-
ment of Health said: “These
excellent results show the new
contract is giving GPs a real
incentive to improve the quality
of care.”
the research at Oxford Univer-
sity, said: “This study shows
statin drugs could be beneficial
in a much wider range of
patients than is currently con-
sidered for treatment.
“What matters most is doc-
tors identify all patients at risk
of a heart attack or stroke –
largely ignoring their present
blood cholesterol level – and
then prescribe a statin at a daily
dose that reduces their choles-
terol substantially.
“Lowering the bad (LDL) cho-
lesterol … with a statin should
reduce the risk of a heart attack
or stroke by at least one third.”
The study, published online in
The Lancet, suggests patients
given a statin would also experi-
ence greater benefits if doctors
aimed to achieve larger reduc-
tions in cholesterol levels.
The analysis also provides
information about the safety of
statins. Earlier studies had
raised concerns they could be
linked with an increased risk of
certain cancers or diseases.
British Heart Foundation
Professor Rory Collins, one of
the study authors, said: “This
work shows clearly that statins
are very safe. There is no good
evidence that statins cause can-
cer and nor do they increase the
risk of other diseases. Although
statins can cause muscle pain or
weakness, our study shows seri-
ous cases are extremely rare.”
Study co-author Professor
John Simes, of Sydney Univer-
sity, said: “The benefits of statin
treatment were seen in all pat-
ient groups, including women,
the elderly, individuals with dia-
betes and those with and with-
out prior heart attack or stroke.”
OPINION 12 DIARY 26 LETTERS 38 TV 39 STARS 43 CROSSWORDS 45 CITY 48 SPORT 52
If you have a story:
Ring 0871 520 2982
If you have a photo:
Ring 0871 520 7171
or send mobile phone pictures
by texting EXPRESS to
07843 500 911
or email to:
expresspix@express.co.uk
DO YOU
HAVE A
STORY
OR A
PHOTO?
We asked in our poll
on Saturday: Should Blair
resign over sending
troops to Iraq?
Yes:97% No: 3%
RESULTS OF
PHONE POLL
3 6 5
8 9 3 4
9 4 5 3 6
9 5 7 1 6
1 5
3 9 5 8 7
7 5 1 4 3
5 2 9 8
1 8 6
Fill in all the squares so that each row, each column,
and 3x3 square contain all the digits from 1 to 9.SUDOKU
7 8 6 1 9 3 5 2 4
2 1 9 5 7 4 8 6 3
5 3 4 8 2 6 9 7 1
4 9 7 2 1 5 6 3 8
8 2 5 6 3 7 1 4 9
3 6 1 9 4 8 7 5 2
9 7 2 3 5 1 4 8 6
6 4 3 7 8 9 2 1 5
1 5 8 4 6 2 3 9 7
Yesterday’s solution
This is the Quick Sudoku
for the solver in a hurry.
For today’s solution call:
0905 072 0051
For extra clues call:
0905 072 0052
(calls cost 75p per minute
from a BT landline)
Another Sudoku: PAGE 45
● The Daily Express Sudoku Book is available from October 3 at £5.99 (p&p included for UK). Don’t miss out,
pre-order your copy now! Send cheque or postal order to Express Bookshop, PO Box 200, Falmouth, Cornwall
TR11 4WJ or call 0871 434 6091, or you can order on-line at www.expressbookshop.com
LOTTO NUMBERS
WEDNESDAY LOTTO
September 21
2 23 33 35 40 42 25
Estimated jackpot is £2.8m
Extra: 2 13 18 25 28 47
Estimated jackpot is £8.7m
T-Ball: 5 14 16 18 32 2
DAILY PLAY
Saturday
1 7 8 12 21 23 24
SATURDAY LOTTO
Sept 24
2 11 27 29 36 44 15
Estimated jackpot is £8.4m
Extra: 3 11 16 34 40 45
Jackpot is £9m this week
T-Ball: 6 8 20 26 34 5
EUROMILLIONS
September 23
1 26 31 34 47
Lucky star numbers: 4 9
Amsterdam . . .c 17 63
Athens . . . . . . .f 25 77
Berlin . . . . . . . .sh 16 61
Cape Town . . . .s 16 61
Corfu . . . . . . . .f 25 77
Dublin . . . . . . .c 18 64
Faro . . . . . . . . .s 24 75
Geneva . . . . . .f 20 68
Gibraltar . . . . . .f 23 73
Guernsey . . . . .s 17 63
Los Angeles . . .f 21 70
Madrid . . . . . . .s 22 72
Malaga . . . . . . .f 24 75
Mallorca . . . . . .s 24 75
Malta . . . . . . . .s 25 77
Miami . . . . . . . .f 31 88
Moscow . . . . . .c 14 57
New York . . . . .c 22 72
Nice . . . . . . . . .s 24 75
Paris . . . . . . . . .c 20 68
Rome . . . . . . . .s 23 73
Stockholm . . . .s 20 68
Sydney . . . . . .r 19 66
Tel Aviv . . . . . . .s 34 93
Tenerife . . . . . .s 28 82
Summary: Showers in places.
World yesterday
Europe today
C F A windy day with rain spreading
eastwards across most areas,
but the south and east of England
will have a fine start.
Outlook tomorrow:
North West East Anglia
Breezy with scattered
showers, locally heavy.
A fresh south-westerly wind.
High 18C (64F).
Early rain will clear to leave
sunny spells. A risk of showers
later. Moderate south-westerly
winds. High 20C (68F).
Northern Ireland London/South East
Sunny intervals and
showers. Cool and breezy
with a strong south-westerly
breeze. High 16C (61F).
After a cloudy start with
rain in places, it will soon
become drier with spells
of sunshine. High 21C (70F).
Wales South
Some sunshine at times,
but with scattered showers,
frequent in the west. Fresh
winds. High 18C (64F).
A cloudy start in places,
but turning brighter with
sunny spells. Afternoon
showers. High 20C (68F).
Midlands South West
Sunny spells, but with a
few showers from the west.
Moderate south-westerly
winds. High 20C (68F).
Some sunshine at times,
but with occasional showers.
Moderate westerly winds.
High 19C (66F).
North East/Yorks Channel Isles
A bright day with sunny spells,
but breezy with the chance
of a shower A fresh breeze.
High 18C (64F).
Early rain will soon clear
to leave sunny intervals,
but with a risk of a shower.
High 18C (64F).
Scotland Sea
Gusty at times with frequent,
heavy showers in the west.
Drier and brighter further
east. High 18C (64F).
North Sea: moderate.
Irish Sea: moderate.
Channel: moderate.
Rain 19C/66F
Amsterdam
Fair 19C/66F
Brussels
Fair 22C/72F
Frankfurt
Fair 23C/73F
Geneva
Fair 22C/72F
Paris
Fair 25C/77F
Rome
Warmest: Chivenor 19C (66F).
Coldest: Aboyne 2C (36F).
Wettest: S’th Uist Range 0.55in.
Sunniest: Beccles 5.0hr.
BRITAIN EXTREMES:
(24 hours to 2pm yesterday)
c-cloudy, dr-drizzle, f-fair, fg-fog,
g-gales, h-hail, hz-haze, m-mist,
r-rain, sh-showers, s-sun,
sn-snow, sl-sleet, th-thunder,
Around Britain
HIGH TIDE: London Bridge: 9.03am, 10.00pm
Liverpool: 7.14am, 7.53pm
Greenock: 8.31am, 7.42pm
Dover: 7.04am, 8.01pm
Belfast 0.0 0.13 9 17
Birmingham 2.0 0.00 9 17
Bristol 2.3 0.00 10 19
Cardiff 3.0 0.01 11 20
Edinburgh 2.0 0.04 7 18
Glasgow 0.9 0.94 10 15
Leeds ** ** 9 16
London 3.4 0.00 12 18
Manchester 1.1 0.00 8 17
Newcastle ** 0.00 10 17
Norwich 3.7 0.04 7 16
Oxford 1.5 0.00 7 17
CITIES
Sun Rain Temp
(hr) (in) (min)(max)
Lighting-up times Yesterday
Belfast 7.10pm-7.20am
Birmingham 6.54pm-7.03am
Bristol 6.57pm-7.06am
Glasgow 7.03pm-7.14am
London 6.47pm-6.56am
Manchester 6.55pm-7.05am
Newcastle 6.52pm-7.03am
MOON rises: –, Sets: 5.14pm
SUN rises London: 6.55am, Sets: 6.47pm
Manchester rises: 7.03am, Sets: 6.55pm
Moon, sun and tides
25
30
30
20
15
13
13
16
18
18
18
19
17
1818
20
20
20
21
Weather today
Miami 31C 88F / Moscow 14C 57F
All forecasts and maps provided by PA WeatherCentre
Daily Express Tuesday September 27 20052
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

8. Quality and Outcomes Framework
later reaction
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

9. Key questions
since QOF was parachuted in Primary Care
What were levels of achievement when QOF introduced?
Did they change over time?
What was happening pre-QOF? Was there an increasing trend?
Practice characteristics asssociated with high performance? Size?
Gap between practices in affluent and deprived areas?
Exception reporting rates and evidence for practice ‘gaming’?
What happened to non-incentivised aspects of care?
Did the intervention effect vary by population group?
What about patient satisfaction?
Various computer systems used. Does system choice affect care?
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

10. Timeline
1998 2014
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
2003
QuIP tp #2
2005
QuIP tp #3
2009
GP Patient
Satisfaction Survey
01/04/2004
QOF introduced
1998 - 2007
QuIP
1998
QuIP time point #1
2005
QOF y1*
2008
GPRD (CPRD)
access
2007
QuIP tp #4
2013
QOF y9
2004 - 2014
QMAS
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

11. Timeline
1998 2014
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
2003
QuIP tp #2
2005
QuIP tp #3
2009
GP Patient
Satisfaction Survey
01/04/2004
QOF introduced
2005
Where's the sun?
1998 - 2007
QuIP
1998
QuIP time point #1
2005
QOF y1*
2008
GPRD (CPRD)
access
2007
QuIP tp #4
2013
QOF y9
2004 - 2014
QMAS
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

12. Levels of achievement in the first year (2004-5)
All 8000+ English practices
Median reported achievement
was 83.4% (IQR: 78.2-87.0%)
Very high achievement levels
Small effects of
sociodemographic
characteristics
Exception reporting not
extensive but the strongest
predictor
A few practices scored high by
excluding large numbers of
patients
The new engl and jour nal of medicine
n engl j med 355;4 www.nejm.org july 27, 2006 375
special article
Pay-for-Performance Programs in Family
Practices in the United Kingdom
Tim Doran, M.P.H., Catherine Fullwood, Ph.D., Hugh Gravelle, Ph.D.,
David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D., Urara Hiroeh, Ph.D.,
and Martin Roland, D.M.
From the National Primary Care Research
and Development Centre, University of
Manchester, Manchester, United King-
dom. Address reprint requests to Dr.
Doran at the National Primary Care Re-
search and Development Centre, William-
son Bldg., University of Manchester, Man-
chester M13 9PL, United Kingdom, or at
tim.doran@manchester.ac.uk.
N Engl J Med 2006;355:375-84.
Copyright © 2006 Massachusetts Medical Society.
Abstr act
Background
In 2004, after a series of national initiatives associated with marked improvements
in the quality of care, the National Health Service of the United Kingdom introduced
a pay-for-performance contract for family practitioners. This contract increases exist-
ing income according to performance with respect to 146 quality indicators cover-
ing clinical care for 10 chronic diseases, organization of care, and patient experience.
Methods
We analyzed data extracted automatically from clinical computing systems for 8105
family practices in England in the first year of the pay-for-performance program
(April 2004 through March 2005), data from the U.K. Census, and data on charac-
teristics of individual family practices. We examined the proportion of patients deemed
eligible for a clinical quality indicator for whom the indicator was met (reported
achievement) and the proportion of the total number of patients with a medical con-
dition for whom a quality indicator was met (population achievement), and we used
multiple regression analysis to determine the extent to which practices achieved high
scores by classifying patients as ineligible for quality indicators (exception reporting).
Results
The median reported achievement in the first year of the new contract was 83.4 per-
cent (interquartile range, 78.2 to 87.0 percent). Sociodemographic characteristics of
the patients (age and socioeconomic features) and practices (size of practice, num-
ber of patients per practitioner, age of practitioner, and whether the practitioner was
medically educated in the United Kingdom) had moderate but significant effects on
performance. Exception reporting by practices was not extensive (median rate, 6 per-
cent), but it was the strongest predictor of achievement: a 1 percent increase in the
rate of exception reporting was associated with a 0.31 percent increase in reported
achievement. Exception reporting was high in a small number of practices: 1 percent
of practices excluded more than 15 percent of patients.
Conclusions
English family practices attained high levels of achievement in the first year of the
new pay-for-performance contract. A small number of practices appear to have
achieved high scores by excluding large numbers of patients by exception reporting.
More research is needed to determine whether these practices are excluding pa-
tients for sound clinical reasons or in order to increase income.
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

13. Trends of prior achievement and 2004-5 expectations
Representative sample of 42 practices
Quality of care for CHD,
asthma and diabetes improved
between 2003 and 2005,
continuing the earlier trend
Increase in rate of
improvement significant only
for asthma and diabetes
Better recording or care?
Practices were preparing in
2003, in anticipation of the
scheme launch?
Gains from the scheme
become questionable, from a
cost-effectiveness perspective
n engl j med 357;2 www.nejm.o182
center Manchester National Health Service.
Data Collection
Trained research staff extracted the data to assess
the quality of clinical care for the categories of
coronary heart disease (15 clinical indicators),
asthma (12 clinical indicators), and type 2 dia-
betes (21 clinical indicators). Data were collected
from both computerized and handwritten medi-
tients
pract
have
for up
practi
for a
tients
These
acros
Alt
that w
in the
clinic
asthm
incen
pared
were
cial in
we ex
this d
clear
vided
Statis
An ov
puted
and 2
heart
comp
cators
divide
that p
repre
provid
100. W
our p
for th
90
MeanPractice-QualityScore(%)
80
85
70
60
55
75
65
0
1998 1999 2000 2001 2002 2003 2004 2005
CHD
Asthma
Diabetes
Year
AUTHOR:
FIGURE:
JOB:
4-C
H/T
RETAKE
SIZE
ICM
CASE
EMail Line
H/T
Combo
Revised
AUTHOR, PLEASE NOTE:
Figure has been redrawn and type has been reset.
Please check carefully.
REG F
Enon
1st
2nd
3rd
Campbell
1 of 1
07-12-07
ARTIST: ts
35702 ISSUE:
22p3
Figure 1. Mean Scores for Clinical Quality at the Practice Level for Coronary
Heart Disease, Asthma, and Type 2 Diabetes, 1998 to 2005.
The quality of care for coronary heart disease (CHD), asthma, and type 2
diabetes was improving between 1998 and 2003, before the introduction of
pay for performance. The rate of improvement in quality of care increased
significantly for diabetes and asthma between 2003 and 2005, after the in-
troduction of pay for performance; the rate for coronary heart disease,
which was increasing most rapidly before pay for performance, continued
at the same rate after pay for performance was introduced.
The New England Journal of Med
Downloaded from nejm.org at UNIV OF MANCHESTER JOHN RYLANDS LIB on September
Copyright © 2007 Massachusetts Medical Society
spe ci a l r ep or t
The new engl and jour nal of medicine
Quality of Primary Care in England
with the Introduction of Pay for Performance
Stephen Campbell, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D.,
Elizabeth Middleton, M.Sc., Bonnie Sibbald, Ph.D., and Martin Roland, D.M.
In 2004, the United Kingdom committed £1.8 bil-
lion ($3.2 billion) to a new pay-for-performance
contract for family practitioners.1 During the first
year, the levels of achievement exceeded those an-
ticipated by the government, with an average of
83.4% of the available incentive payments claimed.2
However, the quality of care in English family
practices had already begun to improve in response
to a wide range of initiatives,3-6 including nation-
al standards for the treatment of major chronic
diseases and a national system of inspection (Ta-
before the pay-for-performance program was in-
troduced and at one time point (2005) after its
introduction. A validated set of criteria was used
to assess quality in the management of three
chronic conditions: asthma, coronary heart dis-
ease, and type 2 diabetes. Because some clinical
indicators — the measures of the quality of clini-
cal care — were not rewarded with financial pay-
ments in the 2004 pay-for-performance program,
the study design also permitted a comparison of
trends in the quality of care for indicators forKontopantelis (IPH) Variation and financial incentives 25 September 2013

14. Performance over time and its variation
All 8000+ English practices
Median overall
achievement was 85.1%,
89.3% and 90.8% in first 3y
Median achievement
increased by 4.4% for
quintile of most affluent and
by 7.6% for most deprived
Gap in median
achievement narrowed from
4.0% to 0.8%
Financial incentive
schemes can contribute to
the reduction of inequalities
Information about practice and patient characteristics
was taken from the 2006 general medical statistics
database, which is maintained by the Department of
Health. Practices were grouped into quintiles of equal
size on the basis of the level of area deprivation in the
census super-output area (a standard, stable unit of
geography used in the UK for statistical analysis; average
population 7200) where they were located, with data from
the Index of Deprivation 2004.20
We calculated the odds of
practices from each quintile being in the top and bottom
performing 5% of practices with respect to achievement
and rates of exclusion by logistic regression. We estimated
the associations of practice-level characteristics with
practice achievement, exclusion of patients, and changes
in these outcomes with multiple linear regressions.
These analyses controlled for missing indicators,
heterogeneity of variance, and clustering of practices,
and we made checks on the robustness of the results to
model specifications (webappendix). All variables were
divided by their standard deviations, thus regression
coefficients show the increase in standard deviations of
the outcome for one standard deviation increase in
predictor variables. All statistical analyses were done with
Stata software (version 9).
Achievement data for 2004–05, 2005–06, and 2006–07
were available for 8277 general practices in England.
Practices were excluded from the study if they had fewer
(164 practices), complete exclusion data were not available
(172 practices), or if the practice population changed in
size by 25% or more (258 practices). Our main results are
drawn from 7637 practices, providing care for more than
49 million patients. We undertook subanalyses for
excluded practices (webappendix).
Role of the funding source
There was no funding source for this study. The
corresponding author had full access to all the data in the
study and had fi nal responsibility for the decision to
submit for publication.
Results
The median overall reported achievement—the propor-
tion of patients who were deemed eligible by the prac-
tices for whom the targets were achieved—was 85·1%
(IQR 79·0–89·1) in year 1, 89·3% (86·0–91·5) in year 2,
and 90·8% (88·5–92·6) in year 3. Increases in
achievement between years were significant (p<0·0001
in all cases). Although average levels of achievement in-
creased over time, variation in achieve ment
diminished.
In year 1, progressively lower levels of achievement
were associated with increased levels of area deprivation
(fi gure 1). Median achievement ranged from 86·8%
(IQR 82·2–89·6) for quintile 1 (least deprived) to 82·8%
Overallreportedachievement(%)
Year 1 (2004–05)
0
20
40
60
80
100
Year 2 (2005–06) Year 3 (2006–07)
Quality and outcome framework year
Quintile 1
Quintile 2
Quintile 3
Quintile 4
Quintile 5
Figure 1: Distribution of scores for overall reported achievement by deprivation quintile for year 1 (2004–05) to year 3 (2006–07)
Central line shows median achievement and box shows interquartile range; whiskers represent range of achievement scores. Circles represent statistical outliers—
ie, individual practices with achievement scores outside the range: first quartile–(1·5×IQR) to third quartile+(1·5×IQR).
most deprived
most affluent
Articles
Effect of financial incentives on inequalities in the delivery of
primary clinical care in England: analysis of clinical activity
indicators for the quality and outcomes framework
Tim Doran, Catherine Fullwood, Evangelos Kontopantelis, David Reeves
Summary
Background The quality and outcomes framework is a financial incentive scheme that remunerates general practices
in the UK for their performance against a set of quality indicators. Incentive schemes can increase inequalities in the
delivery of care if practices in affluent areas are more able to respond to the incentives than are those in deprived
areas. We examined the relation between socioeconomic inequalities and delivered quality of clinical care in the first
3 years of this scheme.
Methods We analysed data extracted automatically from clinical computing systems for 7637 general practices in
England, data from the UK census, and data for characteristics of practices and patients from the 2006 general medical
statistics database. Practices were grouped into equal-sized quintiles on the basis of area deprivation in their locality.
We calculated overall levels of achievement, defined as the proportion of patients who were deemed eligible by the
Lancet 2008; 372: 728–36
Published Online
August 12, 2008
DOI:10.1016/S0140-
6736(08)61123-X
See Comment page 692
National Primary Care Research
and Development Centre,
University of Manchester,
Manchester, UK (T Doran MD,
C Fullwood PhD,
E Kontopantelis PhD,
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

15. Trends of prior achievement and 2006-7 expectations
Representative sample of 42 practices
By 2007 the rate of
improvement had slowed for
all three conditions
Quality of care for
non-incentivised aspects
declined for asthma and CHD
No significant changes on
access to care or on
interpersonal aspects of care
Continuity of care reduced
immediately after the
introduction of the scheme
QOF accelerated short-term
improvements
The new engl and jour nal of medicine
incentives as compared with those that were not.
in the post-introductio
pre-introduction period
ly (P = 0.06). However,
cant when calculated
strapping method (P =
(P = 0.03), and in abso
ity score for aspects of
were not linked to ince
whereas the quality sco
to incentives increased
ate effect of pay for p
between care that wa
linked with incentives
subsequently diverged
vs. pre-introduction pe
duction period vs. intr
with the mean score f
to incentives declining
score for care that wa
creasing. Trends in d
at any time according
linked to incentives.
Communication, Wait
and Continuity of Ca
The percentages of pat
within 48 hours, as w
the physician-commu
significant changes in
declined significantly
pay for performance (
this lower level (Table
Estimated Overall Eff
for Performance
For outcomes in whic
pay for performance a
improvement, we used
rupted time-series ana
of the increase in sco
from the trend in th
(back-transforming th
analysis, with estimat
As compared with the
ment based on the pre-
for-performance schem
22p3
90
Score
85
80
70
65
55
75
60
0
1998 1999 2000 2001 2002 2003 2007200620052004
Coronary
heart disease
Asthma
Diabetes
Year
B
A
AUTHOR:
FIGURE:
JOB:
4-C
H/T
RETAKE
SIZE
ICM
CASE
EMail Line
H/T
Combo
Revised
AUTHOR, PLEASE NOTE:
Figure has been redrawn and type has been reset.
Please check carefully.
REG F
Enon
1st
2nd
3rd
Campbell
1 of 2
xx-xx-09
ARTIST: ts
360xx ISSUE:
100
Score
90
80
60
50
20
70
40
30
0
1998 1999 2000 2001 2002 2003 2007200620052004
Communication
with physicians
Continuity of care
Able to get an appointment
within 48 hr (particular doctor)
Able to get an appointment
within 48 hr (any doctor)
Year
Figure 1. Mean Scores for the Quality of Care at the Practice Level, 1998–2007.
Panel A shows scores for the quality of care provided for coronary heart
disease, asthma, and diabetes. Quality scores range from 0% (no quality
indicator was met for any patient) to 100% (all quality indicators were met
for all patients). Panel B shows scores for patients’ perceptions of commu-
nication with physicians, access to care, and continuity of care. Communi-
cation was assessed by asking seven questions, with the answers scored on
a six-point scale ranging from “very poor” to “excellent”; continuity of care
was assessed with the use of the same six-point scale and a single question:
“How often do you see your usual doctor?” Access to care was scored as
the percentage of patients who reported that they were able to get an ap-
pointment within 48 hours. All scores were rescaled to range from 0 to 100.
special article
The new engl and jour nal of medicine
Effects of Pay for Performance
on the Quality of Primary Care in England
Stephen M. Campbell, Ph.D., David Reeves, Ph.D., Evangelos Kontopantelis, Ph.D.,
Bonnie Sibbald, Ph.D., and Martin Roland, D.M.
From the National Primary Care Research
and Development Centre, University of
Abstr act
Background
A pay-for-performance scheme based on meeting targets for the quality of clinical
care was introduced to family practice in England in 2004.
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

16. Does the size of the practice matter?
All 8000+ English practices
In y1 smallest practices had
the lowest median reported
achievement rates
Performance improved over
time; smallest practices
improved at the fastest rate
Caught up by y3 but displayed
more variation in performance
Small practices represented
among best and worst
QOF reduced variation in
performance and differences
between large and small
practices
e339
proportions over time.
Reported achievement
The median overall reported achievement, the
proportion of patients deemed appropriate by the
practice for whom the targets were achieved, was
85.2% in year 1, 89.3% in year 2, and 90.9% in
year 3. Increases in achievement between years
were statistically significant (P<0.005 in all cases).
Median reported achievement in year 1 varied with
patients, year 1 (2004–2005)
to year 3 (2006–2007).
100
80
60
40
20
0
2004/2005 2005/2006 2006/2007
QOF year
list size 1000–1999
2000–2999
3000–3999
4000–5999
6000–7999
8000–9999
10000–11999
12000 or more
Overallreportedachievement
Figure 2. Distribution of
practice scores for overall
reported achievement by
number of patients, year 1
(2004–2005) to year 3
(2006–2007).
Central white line shows median scores and box shows interquartile range (IQR); whiskers
represent range of scores. Circles represent statistical outliers — that is, individual practices
with points scores outside the range: first quartile — (1.5 × IQR) + (1.5 × IQR).
ABSTRACT
Background
Small general practices are often perceived to provide
worse care than larger practices.
Aim
To describe the comparative performance of small
INTRODUCTION
Small general practices in the UK, particularly those
that are single handed, are often accused of
providing poor-quality care. The 2000 NHS Plan cited
a need to ‘confirm that single-handed (solo)
practices are offering high standards’.1
The Shipman
Inquiry identified advantages and disadvantages
T Doran, S Campbell, C Fullwood, et al
Performance of small general
practices under the UK’s Quality
and Outcomes Framework
Tim Doran, Stephen Campbell, Catherine Fullwood,
Evangelos Kontopantelis and Martin RolandBJGP 2010
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

17. Changes in patient experiences
42 practices, random samples of chronic condition and all patients
No changes in 2003-7
quality of care for
communication, nursing
care, coordination &
overall satisfaction
Aspects of access
improved for chronic
disease patients only
Both samples seeing their
usual GP less & less
satisfied in continuity
Related to incentives to
provide rapid
appointments
ANNALS OF FAMILY MEDICINE ✦
WWW.ANNFAMMED.ORG ✦
VOL. 8, NO. 6 ✦
NOVEMBER/DECEMBER 2010
502
of-access items (P >.05), including ability to see a
particular physician or any physician within 48 hours. Patients Randomly Sampled From
Registered Lists
Results for patients randomly sampled from practice
lists matched those for patients with chronic illness in
all aspects except the speed-of-access items: no signifi-
cant changes over time were observed in any of these
items. The only significant changes from 2003 to 2007
were in regard to continuity of care, with a reduction
in how often patients reported being able to see their
Table 2. Response Rates for Patient Surveys
Patient Group
2003
% (n)
2005
% (n)
2007
% (n)
Chronic illness samples 55 (1,092) 52 (1,040) 50 (922)
Random samples of
registered patients
47 (3,873) 45 (3,601) 37 (3,104)
Table 3. Summary of Practice Mean GPAQ Scale and Individual Item Scores 2003, 2005, and 2007,
for Cross-Sectional Samples of Patients With Chronic Illness and Random Samples of Adult Patients
GPAQ Scale and Item
Samples of Patients
With Chronic Illness
Mean (SD)a
Samples of Randomly
Selected Patients
Mean (SD)a
2003 2005 2007 2003 2005 2007
Communication scale 74.5 (10.1) 74.5 (9.3) 76.0 (8.5) 69.5 (10.0) 68.4 (9.4) 69.9 (8.9)
Coordination scale 72.1 (7.4) 71.0 (5.8) 73.0 (5.9) 67.0 (5.8) 68.6 (5.1) 68.3 (6.1)
Nursing care scale 76.0 (6.1) 76.1 (6.0) 75.3 (8.1) 73.9 (6.2) 73.4 (5.3) 72.9 (7.1)
Overall satisfaction scale 81.6 (8.0) 80.6 (8.6) 81.3 (7.7) 75.2 (9.2) 74.7 (10.6) 75.6 (7.9)
Item: In general, how often do you see your
usual doctor (continuity of care)?
77.2 (8.0) 74.9 (9.4) 70.4 (9.6) 68.2 (13.1) 62.8 (13.5) 62.5 (11.6)
Item: Rating of how often patients get to see
their usual doctor (rating of continuity of care)
71.5 (9.8) 69.1 (9.9) 67.4 (11.5) 64.6 (13.1) 61.0 (12.2) 61.2 (10.4)
Item: Do you get an appointment with a particu-
lar doctor within 48 hours?
36.9 (29.5) 38.7 (26.3) 37.0 (22.6) 33.4 (26.2) 35.9 (25.1) 32.3 (21.0)
Item: Rating of how quickly an appointment
can be made with a particular doctor in the
practice
57.3 (17.7) 58.0 (14.1) 56.6 (15.6) 50.0 (18.6) 52.2 (16.0) 53.0 (13.6)
Item: Do you get an appointment with any doc-
tor within 48 hours?
64.5 (23.9) 68.0 (22.6) 67.2 (19.9) 61.5 (24.2) 63.4 (21.0) 63.8 (22.4)
Item: Rating of how quickly an appointment can
be made with any doctor in the practice
65.2 (15.2) 64.5 (13.8) 65.5 (14.5) 59.2 (17.2) 61.2 (14.2) 62.4 (13.1)
Item: If you need an urgent appointment can you
get one on the same day?
81.8 (17.7) 81.2 (15.5) 82.2 (16.8) 79.6 (17.1) 78.2 (17.5) 79.4 (18.8)
GPAQ=General Practice Assessment Questionnaire.
Note: See the Appendix for a description of how the scales were scored.
a
Figures relate to raw practice-level scores (mean and standard deviation of practice means).
13/09/2013 Changes in Patient Experiences of PrimaryCare During Health Service Reforms
+
The Annals of Family Medicine
www.annfammed.org
doi: 10.1370/afm.1145
Ann Fam Med November 1, 2010 vol. 8 no. 6 499-506
Changes in Patient Experiences of
Primary Care During Health Service
Reforms in England Between 2003 and
2007
Stephen M. Campbell, PhD1, Evangelos Kontopantelis, PhD1,
David Reeves, PhD1, Jose M. Valderas, PhD1, Ella Gaehl, MPhil1,
Nicola Small, MPhil1 and Martin O. Roland, DM2
Author Affiliations
CORRESPONDING AUTHOR: Stephen Campbell, PhD, National Primary Care
Research and Development Centre, University of Manchester, Williamson Bldg,
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

18. Patient satisfaction
2007-8 GP Access Survey, all English practices
Young, Asian, working FT, with
long commuting times: lowest
levels of satisfaction and
experience of access
Ability to take time off to visit
GP eliminated the
disadvantage in access
Patients in small practices
more positive for all aspects of
access; except opening hours
Positive access to care
associated with higher QOF
scores and slightly lower rates
of emergency admission
0
20
40
60
80
100
%ofpositiveresponses
<1
1−2
2−3
3−4
4−6
6−88−1010−12>=12
Phone access
0
20
40
60
80
100
%ofpositiveresponses
<1
1−2
2−3
3−4
4−6
6−88−1010−12>=12
Appointment within 2 days
0
20
40
60
80
100
%ofpositiveresponses
<1
1−2
2−3
3−4
4−6
6−88−1010−12>=12
Advance appointment (>2 days)
0
20
40
60
80
100
%ofpositiveresponses
<1
1−2
2−3
3−4
4−6
6−88−1010−12>=12
Appointment with a particular GP
Practice size in 1,000s of patients
by practice list size
Satisfaction & positive experience
RESEARCH ARTICLE Open Access
Patient experience of access to primary care:
identification of predictors in a national patient
survey
Evangelos Kontopantelis1*
, Martin Roland2
, David Reeves1
Abstract
Background: The 2007/8 GP Access Survey in England measured experience with five dimensions of access:
getting through on the phone to a practice, getting an early appointment, getting an advance appointment,
Kontopantelis et al. BMC Family Practice 2010, 11:61
http://www.biomedcentral.com/1471-2296/11/61
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

19. Advising NICE on removing indicators
All English practices, 2004/5 to 2009/10
Indicators should be removed on
statistical criteria and the economics
of incentives
Replaced with other indicators to
drive improvement in other areas
It is unknown what will happen to
the withdrawn indicators
BMJ | 24 APRIL 2010 | VOLUME 340 899
ANALYSIS
Increasing numbers of countries are using indi-
cators to evaluate the quality of clinical care,
with some linking payment to achievement.1
For performance frameworks to remain effective
the indicators need to be regularly reviewed. The
frameworks cannot cover all clinical areas, and
achievement on chosen indicators will even-
tually reach a ceiling beyond which further
improvement is not feasible.2 3
However, there
has been little work on how to select indictors for
replacement. The Department of Health decided
in 2008 that it would regularly replace indicators
in the national primary care pay for performance
scheme, the Quality and Outcomes Framework,4
making a rigorous approach to removal a prior-
ity. We draw on our previous work on pay for
performance5 6
and our current work advising
the National Institute for Health and Clinical
Excellence (NICE) on the Quality and Outcomes
Framework to suggest what should be consid-
ered when planning to remove indicators from
a clinical performance framework.
First UK decisions
The Quality and Outcomes Framework cur-
rently includes 134 indicators for which gen-
eral practices can earn up to a total of 1000
points. Negotiations between the Department
of Health and the BMA’s General Practitioners
Committee last autumn led to an agreement to
remove eight clinical indicators worth 28 points
in April 2011 (table 1). The eight indicators are
all process measures and reward actions such as
taking blood pressure or taking blood to measure
cholesterol, glucose, or creatinine concentra-
tions for people with relevant chronic diseases.
The framework rewards the action itself rather
than a clinically informed response to results or
intermediate outcomes such as better control of
blood pressure or cholesterol levels. It is there-
fore not surprising that achievement of these
process indicators is high (median >95% and
interquartile range <4.5%) with little change in
rates or variation across practices since 2005-6,
the second year of the Quality and Outcomes
Framework.
In many schemes, including the Quality and
Outcomes Framework, providers can “except”
certain patients from inclusion in the denomi-
nator figures for an indicator on grounds such as
extreme frailty or contraindications to a specified
drug. Exception reporting rates are also low for
these eight indicators (median <5% and inter-
quartile range <3%).
How to identify when a performance
indicator has run its course
In April 2011 eight clinical indicators will be removed from the UK Quality and Outcomes
Framework. David Reeves and colleagues explain why they were chosen and suggest
a rationale for future decisions
National achievement and exception rates for indicators that are to be removed from the Quality and Outcomes Frameworkin 20117
Indicator (measurement of ) Condition
Median (interquartile range) achievement (%) Median (interquartile range) rate of exceptions (%) Paired
indicator*2005-6 2006-7 2007-8 2005-6 2006-7 2007-8
Bloodpressure Coronaryheartdisease 98.2 (96.7-99.3) 98.5 (97.2-99.5) 98.4(97.1-99.4) 0.8 (0-1.8) 0.8 (0-1.7) 0.7 (0-1.6) Yes
Haemoglobin A1c
Diabetes 97.4 (95.1-98.8) 97.8 (95.9-99.0) 97.7 (96.0-98.9) 2.7 (1.4-4.4) 2.5 (1.3-4.2) 2.4 (1.2-3.9) Yes
Bloodpressure Diabetes 98.8 (97.6-99.7) 99.0 (98.0-100) 98.9 (97.9-99.6) 1.1 (0.3-2.2) 1.0 (0.3-2.1) 1.0 (0.3-2.1) Yes
Serum creatinine Diabetes 96.7 (94.1-98.3) 97.4 (95.3-98.7) 97.4 (95.6-98.7) 1.9 (0.9-3.4) 1.7 (0.8-3.2) 1.6 (0.7-3.1) No
Totalcholesterol Diabetes 96.4 (93.8-98.1) 96.9 (94.8-98.3) 96.8 (94.8-98.2) 2.2 (1.1-3.9) 2.1 (1.0-3.8) 2.0 (0.9-3.6) Yes
Serum creatinine andthyroid
stimulating hormone
Mentalhealth (lithium) 100 (100-100) 100 (100-100) 100 (100-100) 0 (0-0) 0 (0-0) 0 (0-0) No
Bloodpressure Stroke 97.4 (95.1-100) 97.7 (95.8-100) 97.6 (95.8-99.5) 1.3 (0-3.3) 1.2 (0-2.9) 1.1 (0-2.7) Yes
Thyroidfunction Hypothyroidism 96.8 (94.5-98.7) 96.7 (94.5-98.5) 96.5 (94.3-98.3) 0 (0-0.9) 0 (0-0.9) 0 (0-0.8) No
*Paired indicators relate to control of the relevant measure—for example, the indicator that focuses on recording blood pressure in patients with coronary heart disease, is paired with another
indicator that rewards on the basis of the proportion of patients whose last blood pressure reading was ≤150/90 mm Hg.
BMJ | 24 april 2010 | Volume 340 899
tually reach a ceiling beyond which further
improvement is not feasible.2 3
However, there
has been little work on how to select indictors for
replacement. The Department of Health decided
in 2008 that it would regularly replace indicators
in the national primary care pay for performance
scheme, the Quality and Outcomes Framework,4
making a rigorous approach to removal a prior-
ity. We draw on our previous work on pay for
performance5 6
and our current work advising
the National Institute for Health and Clinical
Excellence (NICE) on the Quality and Outcomes
Framework to suggest what should be consid-
ered when planning to remove indicators from
a clinical performance framework.
First UK decisions
The Quality and Outcomes Framework cur-
rently includes 134 indicators for which gen-
eral practices can earn up to a total of 1000
points. Negotiations between the Department
of Health and the BMA’s General Practitioners
­Committee last autumn led to an agreement to
remove eight clinical indicators worth 28 points
in April 2011 (table 1). The eight indicators are
all process measures and reward actions such as
taking blood pressure or taking blood to measure
­cholesterol, glucose, or creatinine concentra-
tions for people with relevant chronic diseases.
The framework rewards the action itself rather
than a clinically informed response to results or
intermediate outcomes such as better control of
blood pressure or cholesterol levels. It is there-
fore not surprising that achievement of these
process indicators is high (median >95% and
interquartile range <4.5%) with little change in
rates or variation across practices since 2005-6,
the second year of the Quality and Outcomes
Framework.
In many schemes, including the Quality and
Outcomes Framework, providers can “except”
certain patients from inclusion in the denomi-
nator figures for an indicator on grounds such as
extreme frailty or contraindications to a specified
drug. Exception reporting rates are also low for
these eight indicators (median <5% and inter-
quartile range <3%).
National achievement and exception rates for indicators that are to be removed from the Quality and Outcomes Frameworkin 20117
Indicator (measurement of ) Condition
Median (interquartile range) achievement (%) Median (interquartile range) rate of exceptions (%) Paired
indicator*2005-6 2006-7 2007-8 2005-6 2006-7 2007-8
Bloodpressure Coronaryheartdisease 98.2 (96.7-99.3) 98.5 (97.2-99.5) 98.4(97.1-99.4) 0.8 (0-1.8) 0.8 (0-1.7) 0.7 (0-1.6) Yes
Haemoglobin A1c
Diabetes 97.4 (95.1-98.8) 97.8 (95.9-99.0) 97.7 (96.0-98.9) 2.7 (1.4-4.4) 2.5 (1.3-4.2) 2.4 (1.2-3.9) Yes
Bloodpressure Diabetes 98.8 (97.6-99.7) 99.0 (98.0-100) 98.9 (97.9-99.6) 1.1 (0.3-2.2) 1.0 (0.3-2.1) 1.0 (0.3-2.1) Yes
Serum creatinine Diabetes 96.7 (94.1-98.3) 97.4 (95.3-98.7) 97.4 (95.6-98.7) 1.9 (0.9-3.4) 1.7 (0.8-3.2) 1.6 (0.7-3.1) No
Totalcholesterol Diabetes 96.4 (93.8-98.1) 96.9 (94.8-98.3) 96.8 (94.8-98.2) 2.2 (1.1-3.9) 2.1 (1.0-3.8) 2.0 (0.9-3.6) Yes
Serum creatinine andthyroid
stimulating hormone
Mentalhealth (lithium) 100 (100-100) 100 (100-100) 100 (100-100) 0 (0-0) 0 (0-0) 0 (0-0) No
Bloodpressure Stroke 97.4 (95.1-100) 97.7 (95.8-100) 97.6 (95.8-99.5) 1.3 (0-3.3) 1.2 (0-2.9) 1.1 (0-2.7) Yes
Thyroidfunction Hypothyroidism 96.8 (94.5-98.7) 96.7 (94.5-98.5) 96.5 (94.3-98.3) 0 (0-0.9) 0 (0-0.9) 0 (0-0.8) No
*Paired indicators relate to control of the relevant measure—for example, the indicator that focuses on recording blood pressure in patients with coronary heart disease, is paired with another
indicator that rewards on the basis of the proportion of patients whose last blood pressure reading was ≤150/90 mm Hg.
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

20. Non-incentivised aspects of care
Sample of 148 representative practices from the CPRD
Achievement rates improved
for most indicators in the
pre-incentive period
Significant initial gains in
incentivised indicators but no
gains in later years
No overall effect on the rate of
improvement for non
incentivised indicators in
2004-5
But by 2006-7 achievement
rates significantly below those
predicted by pre-incentive
trends
Figure
Mean achievement rate of 148 general practices for qua
grouped by activity and whether they were incentivised
mean rate is the mean of the adjusted means for the in
Effect of financial incentives on incentivised and
non-incentivised clinical activities: longitudinal
analysis of data from the UK Quality and Outcomes
Framework
Tim Doran clinical research fellow1
, Evangelos Kontopantelis research associate1
, Jose M Valderas
clinical lecturer 2
, Stephen Campbell senior research fellow 1
, Martin Roland professor of health
services research3
, Chris Salisbury professor of primary healthcare4
, David Reeves senior research
fellow 1
1
National Primary Care Research and Development Centre, University of Manchester, Manchester M13 9PL, UK; 2
NIHR School for Primary Care
Research, Department of Primary Health Care, University of Oxford, Oxford OX3 7LF; 3
General Practice and Primary Care Research Unit, University
of Cambridge, Cambridge CB2 0SR; 4
Academic Unit of Primary Health Care, University of Bristol, Bristol BS8 2AA
Abstract
Objective To investigate whether the incentive scheme for UK general
practitioners led them to neglect activities not included in the scheme.
Design Longitudinal analysis of achievement rates for 42 activities (23
included in incentive scheme, 19 not included) selected from 428
identified indicators of quality of care.
Setting 148 general practices in England (653 500 patients).
Main outcome measures Achievement rates projected from trends in
the pre-incentive period (2000-1 to 2002-3) and actual rates in the first
Introduction
Over the past two decades funders and policy makers worldwide
have experimented with initiatives to change physicians’
behaviour and improve the quality and efficiency of medical
care.1
Success has been mixed, and attention has recently turned
to payment mechanism reform, in particular offering direct
financial incentives to providers for delivering high quality
care.2
In 2004 in the UK the Quality and Outcomes Framework
(QOF) was introduced—a mechanism intended to improve
BMJ 2011;342:d3590 doi: 10.1136/bmj.d3590 Page 1 of 12
Research
RESEARCH
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

21. Reasons for exception reporting patients
All English practices in 2008-9
Median exception rate was 2.7% (IQR 1.9-3.9%) overall and
0.44% (0.14-1.1%) for informed dissent
Common reasons logistical (40.6%), clinical contraindication
(18.7%), patient informed dissent (30.1%)
Higher rates associated with: larger practices, higher deprivation,
failure to secure maximum remuneration in previous year
Cost of the provision relatively low at £0.58 per patient (£31m)
Relatively few patients excluded for informed dissent, suggesting
that the incentivised activities were broadly acceptable to patientsFigures
Fig 1 Proportion of patients exception reported by indicator and reason, 2008-9. For 37 indicators for which reasons for
exception reporting were ascribable (see table 1). Indicators ordered by type of activity (measurement or outcome) and by
rate of exception reporting attributable to informed dissent
BMJ 2012;344:e2405 doi: 10.1136/bmj.e2405 (Published 17 April 2012) Page 11 of 11
RESEARCH
Exempting dissenting patients from pay for
performance schemes: retrospective analysis of
exception reporting in the UK Quality and Outcomes
Framework
OPEN ACCESS
Tim Doran clinical research fellow
1
, Evangelos Kontopantelis research fellow
1
, Catherine Fullwood
research associate
2
, Helen Lester professor of primary care
3
, Jose M Valderas clinical lecturer
4
,
Stephen Campbell senior research fellow
1
1
Health Sciences Research Group-Primary Care, University of Manchester, Manchester M13 9PL, UK; 2
Manchester Academic Health Science
Centre, Central Manchester University Hospitals NHS Foundation Trust, University of Manchester; 3
School of Health and Population Sciences,
University of Birmingham, Birmingham, UK; 4
NIHR School for Primary Care Research, Health Services and Policy Research Group, Department
of Primary Care Health Sciences, University of Oxford, Oxford, UK
Abstract
Objective To examine the reasons why practices exempt patients from
the UK Quality and Outcomes Framework pay for performance scheme
(exception reporting) and to identify the characteristics of general
Conclusions The provision to exception report enables practices to
exempt dissenting patients without being financially penalised. Relatively
few patients were excluded for informed dissent, however, suggesting
that the incentivised activities were broadly acceptable to patients.
BMJ 2012;344:e2405 doi: 10.1136/bmj.e2405 (Published 17 April 2012) Page 1 of 11
Research
RESEARCH
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

22. Changes to the incentives
All English practices, 2004-5 to 2009-10
Four influenza immunisation
indicators for CHD, COPD,
diabetes and stroke
Upper threshold increased by
5% only for CHD in 2006-7
The increase led to increase in
the % of immunised CHD
patients by 0.41% but also to
exception reporting by 0.26%
Making quality targets more
demanding can lead to
improvement in quality of care
but can also have other
consequences
Minimum
threshold
y1/2
Mminimum
threshold
y3/5
Maximum
threshold
y1/2
Maximum
threshold
y3/5
02004006008001000
Numberofpractices
0 20 40 60 80 100
Percentage of patients
08/09 07/08 06/07 05/06 04/05
CHD 12
Reported achievement
Family Doctor Responses to Changes in
Incentives for Influenza Immunization
under the U.K. Quality and Outcomes
Framework Pay-for-Performance
Scheme
Evangelos Kontopantelis, Tim Doran, Hugh Gravelle, Rosalind
Goudie, Luigi Siciliani, and Matt Sutton
Objective. To analyze the effect of setting higher targets, in a primary care pay-for-
performance scheme, on rates of influenza immunization and exception reporting.
© Health Research and Educational Trust
DOI: 10.1111/j.1475-6773.2011.01362.x
RESEARCH ARTICLE
Health Services Research
Health Services Research, 2012
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

23. Patient level diabetes care
Sample of 148 representative practices from the CPRD
In 2004-5 quality improved
over-and-above this
pre-incentive trend by 14.2%
By 2006-7 the improvement
above trend was smaller at 7.3
Levels of care varied
significantly for sex, age, years
of previous care, number of
co-morbid conditions
Recorded quality of primary care for
patients with diabetes in England
before and after the introduction
of a financial incentive scheme:
a longitudinal observational study
Evangelos Kontopantelis,1
David Reeves,1
Jose M Valderas,2,3
Stephen Campbell,1
Tim Doran1
▸ An additional data is
published online only. To view
this file please visit the journal
online (http://bmjqs.bmj.com)
1
Health Sciences Primary Care
Research Group, University of
ABSTRACT
Background The UK’s Quality and Outcomes
Framework (QOF) was introduced in 2004/5,
linking remuneration for general practices to
recorded quality of care for chronic conditions,
years were more modest. Variation in care
between population groups diminished under
the incentives, but remained substantial in some
cases.
ORIGINAL RESEARCH
group.bmj.comon September 13, 2013 - Published byqualitysafety.bmj.comDownloaded from
Recorded QOF care did not vary significantly by area
deprivation before or after the introduction of the
incentivisation scheme. However, the effect of the inter-
vention did vary with area deprivation: patients attend-
ing practices from the most deprived quartile appear to
have gained less from the intervention compared with
patients in the most affluent quartile of practices, by
4.9% in 2004/5 and 3.8% in 2006/7.
There was significant variation in recorded QOF care
by practice diabetes prevalence rates, but the differences
Figure 2 Aggregate patient level Quality and Outcomes
Framework care and predictions based on the
pre-incentivisation trend.
on Sequalitysafety.bmj.comDownloaded from
2000/1 2001/2 2002/3 2003/4 2004/5 2005/6 2006/7
new diagno 44.7 50.4 56.5 65.3 73.4 74.2 74.3
1-4 years 48.4 53.9 59.4 71.1 80.9 83 83.2
5-9 years 46.4 51.9 56.8 69.1 78.7 81.4 81.8
10+ years 45.4 50 55.1 66.7 77.6 79.3 80.4
2000/1 2001/2 2002/3 2003/4 2004/5 2005/6 2006/7
new diagnoses 44.7 50.4 56.5 65.3 73.4 74.2 74.3
1-4 years 48.4 53.9 59.4 71.1 80.9 83 83.2
5-9 years 46.4 51.9 56.8 69.1 78.7 81.4 81.8
10+ years 45.4 50 55.1 66.7 77.6 79.3 80.4
40
45
50
55
60
65
70
75
80
85
90
aggregaterecordedQOFcarescore
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

24. Clinical computer systems
All English practices, 2007-8 to 2010-11
Seven clinical computer
systems consistently active
collectively holding ≈ 99% of
the market share
System choice strongest
predictor of performance on
62 stable care indicators
Differences greatest for
intermediate outcome
indicators
Particular system
characteristics facilitate higher
quality of care, better data
recording or both?
North East
North West
London
West Midlands
Yorkshire & the Humber
South West
East Midlands
East of England
South Central
South East Coast
(85.4,85.7]
(85.1,85.4]
(84.8,85.1]
(84.5,84.8]
(84.2,84.5]
[83.9,84.2]
LV
Vision 3
ProdSysOneX
PCS
Synergy
Practice Manager
Premiere
NOTE: Chart size proportional to number of practices in area
Average practice scores by Strategic Health Authority, 2010−11
Overall population achievement (62 indicators)
and GP systems products
Relationship between quality of care
and choice of clinical computing
system: retrospective analysis of family
practice performance under the UK’s
quality and outcomes framework
Evangelos Kontopantelis,1,2
Iain Buchan,3
David Reeves,1,2
Kath Checkland,1
Tim Doran4
To cite: Kontopantelis E,
Buchan I, Reeves D, et al.
Relationship between quality
of care and choice of clinical
computing system:
retrospective analysis of
family practice performance
under the UK’s quality and
outcomes framework. BMJ
ABSTRACT
Objectives: To investigate the relationship between
performance on the UK Quality and Outcomes
Framework pay-for-performance scheme and choice of
clinical computer system.
Design: Retrospective longitudinal study.
Setting: Data for 2007–2008 to 2010–2011, extracted
from the clinical computer systems of general practices
ARTICLE SUMMARY
Article focus
▪ Practice and patient-level characteristics are
known predictors of quality of care, as measured
by the Quality and Outcomes Framework (QOF)
indicators.
▪ Various general practitioner (GP) clinical com-
Open Access Research
group.bmj.comon September 13, 2013 - Published bybmjopen.bmj.comDownloaded from
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

25. Getting there...
Very high levels of initial
achievent leading to
overpayment
Still confusion about whether
the aim is to reward high
quality of care or to drive
improvement
It appears that the same levels
of care would have been met
eventually; QOF just took us
there quicker
050010001500
Numberofpractices
0 20 40 60 80 100
Percentage of patients
08/09 07/08 06/07 05/06 04/05
Overall, 48+2 (smoking) indicators
Reported achievement
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

26. Getting there...
Reduced inequalities but negatively affected continuity and
aspects of non incentivised care
QOF led to bigger, better organised practices but patients do not
seem to like that
Exception reporting a cheap provision to ensure no patient
discrimination
Small changes in the scheme details can have big effects on
quality of care
The intervention effect varied by patient groups
Strongest predictor of achievement was clinical system
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

27. What’s to be done with it?
good on paper but massive cost
Limit incentives to intermediate
outcome indicators only?
Remove the upper thresholds for
indicators, thereby reducing the
cost and driving improvement?
Re-negotiate the payment
platform for the same reason?
Bigger range of indicators and
only a random set assessed each
year?
Drop the whole scheme and
re-invest in other ventures?
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

28. More questions
What happens when the incentive for an indicator is removed?
What is the effect of the incentivisation on harder outcomes?
mortality
complications (e.g. for diabetes)
What really happens with exceptions and is there any gaming?
Kontopantelis (IPH) Variation and financial incentives 25 September 2013

29. Thank you for listening!
Comments and questions: e.kontopantelis@manchester.ac.uk
Kontopantelis (IPH) Variation and financial incentives 25 September 2013