SlideShare a Scribd company logo
1 of 10
Download to read offline
The formative impact of general practice
appraisals: a post-appraisal
questionnaire survey
Olga Zolle BSc PhD PGCE
Research and Development Manager, NHS Education South Central, Winchester, UK
Mark Rickenbach FRCP FRCGP DCH FHEA PhD
Associate Dean and GP Trainer, NHS Education South Central, Wessex Deanery and Honorary
Fellow, University of Winchester, Winchester, UK
Eileen Gorrod BM FRCGP GP
Appraisal Clinical Lead
NHS Education South Central, Winchester, UK
SUMMARY
Since their national implementation in 2002, GP
appraisals are being extensively applied. There is
a positive link between appraisals and patient
care in hospitals.1
There also appears to be no
uniform model for appraisals,2
which has led to
confusion between their ‘stand-alone’ role as a
formative assessment tool3–5
and their contribu-
tion to summative revalidation.
This is a case study into the formative ele-
ment of appraisals as carried out at a primary
care trust in Wessex. We offer an insight into
the diversity of learning outcomes and develop-
mental potential of the formative element of
appraisals (excluding revalidation) attained
through the analysis of emerging themes from
responses to a purposely designed question-
naire aimed at appraisees (questions appear in
the tables).
To our knowledge this is one of the few
research articles that deal with the formative edu-
cational element of GP appraisals.
INTRODUCTION
Appraisals were originally designed to act as
developmental and hence formative tools for
assessment and not as performance management
tools.3,6–,8
They provide opportunities for confiden-
tial discussion of the problems and solutions
Education for Primary Care (2009) 20: 445–54 # 2009 Radcliffe Publishing Limited
WHAT IS ALREADY KNOWN IN THIS AREA
. The formative element of appraisals provides a unique opportunity for discussion of learning needs and
outcomes with a trained peer.
. The extent of the impact of this form of appraisal remains unclear.
WHAT THIS WORK ADDS
. Main impacts of appraisals identified by appraisees are an increase in their confidence, feeling valued in
their jobs, improvement in overall patient care and contributing to a delay in retirement age.
. It emphasises the need for purpose-trained appraisers. The main clear, challenging and achievable
learning objectives analysed as a result of the study were in clinical skills, running of a practice and life
coaching, suggesting a need to train appraisers to provide advice in these areas.
SUGGESTIONS FOR FUTURE RESEARCH
. Comparison of learning outcomes arising from appraisals with and without revalidation.
. Provide further insight into the training of appraisers, particularly on the formative element of the
appraisal through an in-depth analysis of the types of learning outcomes that might stem from it.
Keywords: appraisal, appraisal model, formative assessment, peer assessment, revalidation
encountered in general practice and are intended
to help identify appropriate challenging learning
needs that are translated into outcomes. They may
therefore influence and improve standards in a
non-judgemental manner.3,7
Research shows that
GP appraisals act as a medium for discussion of
issues arising from everyday practice and are per-
ceived as beneficial in reflecting individual oppor-
tunities and overall practice performance.3,7,9
The aims of appraisals are in contrast with the
General Medical Council’s (GMC’s) revalidation
system of summative assessments that consider
whether an individual is fit for practice.6,10
Revalid-
ation has imposed outcomes and is a matter of
public record.11,12
Appraisal is being put forward
as an indicator for revalidation, the application of
clinical governance systems and also to track
Good Medical Practice.13
However, Middlemas
and Siriwardena (2003)5
performed a cross-sec-
tional study amongst GPs and concluded that
appraisals ‘might not detect incompetent doctors’.
The problem is how these two very distinct sys-
tems (one is confidential and formative while the
other is in the public domain and summative) can
be brought together in an effective manner with-
out disturbing the unique nature, effectiveness
and positive educational impact that appraisals
may provide for GP practice.14
Another argument
for joining appraisals with revalidation is cost
effectiveness:15,16
Bruce et al (2004)17
compared
two distinct methods for revalidation, one linked
to appraisals and one that is independent. They
report that ‘peer or educational support will be
required’ for both methods.
METHODS
At the time of the study the PCT is working
towards the standards set in Assuring the Quality
of Medical Appraisal by the NHS Clinical Govern-
ance Support Group – July 2005.18
In contrast to
other appraisal models currently maintained
across the country, this PCT uses a model for
appraisal set clearly apart from revalidation.
Although the personal development plan is revis-
ited during the appraisal this is not considered to
be a formal shared outcome.
All the appraisers at this PCT attend:
. regular developmental meetings
. a developmental day
. other non-compulsory conferences and events
related to their training for example the National
Association of Primary Care Educators’ appraisal
conference held annually.
The developmental day takes place once every 18
months. The developmental meetings last two
hours and occur three times a year. The meetings
are attended by a GP tutor who is able to provide
an overview of the appraisal process across the
strategic health authority (SHA) and nationally.
These meetings act as a forum for the exchange
of ‘best practice’ ideas, updating and improve-
ment of the process. At the time of writing, speci-
fic emphasis is being placed on the issue of
‘moving the appraisee on’.
For the purpose of this study, all the GPs who
were appraised over the period of a year (from
April 2005 to April 2006) were asked to complete a
questionnaire regarding the educational impact
and quality of the appraisal. The appraisees
selected the appraiser from a list provided by the
PCT. The appraiser then contacted the appraisee
to confirm the date and time of appraisal as well
as providing an outline of the process. The apprai-
sal process ensured that the personal develop-
ment plan followed the SMART criteria of specifi-
city, measurability, achievability, relevance and
time bound. The confidential paperwork needed to
provide support and evidence for the process was
available to the appraiser any time over two weeks
prior to the appraisal. Half a day was allowed for
the appraisal interview and feedback. A summary
of the appraisal interview (Form 4) was completed
by both appraisee and appraiser together. The
summary information was collated and fed back
anonymously to the appraisers group and PCT
highlighting the educational outcomes and identi-
fying areas for future improvement.
The qualitative and quantitative questionnaire
developed specifically for this project explores
the attitudes and feelings of appraisees before,
during and after the appraisal. It was provided
immediately following the appraisal. A copy of the
questionnaire may be found in the Appendix. All
participants were informed about the aims of the
study before receiving their consent. They were
also informed of their rights to withdraw from the
study without providing a reason (Human Rights
Act 1998). All the anonymous questionnaires were
then sent to a single named person and were
anonymised, and treated as confidential in line
with the requirements of the Data Protection Act
1998. Further information on perceived potential
learning outcomes was derived from a thematic
analysis of the summaries found in Form 4. Con-
sent was also sought for the use of these data.
The qualitative data were analysed based on the
emerging themes from the responses to the ques-
tions. The quantitative data were analysed using
EXCEL software. The quantitative and qualitative
data provided the following breakdown of themes:
. feelings prior to the appraisal
. quality of feedback received from the appraiser
. perceived impact of the appraisal
. potential learning outcomes from Form 4s
. qualitative analysis on general comments about
the appraisal.
The paper is written to reflect the above.
The researcher and main author (OZ) is not a GP
and has no involvement in the GP appraisal pro-
cess. The data were analysed by OZ and the
emerging themes were corroborated with MR who
446 O Zolle, M Rickenbach and E Gorrod
is a GP. EG collected the questionnaires and
anonymised them. SS provided some informal
feedback in the set-up stage. There may be some
bias in terms of using MR’s experience of apprais-
als to inform the paper. In this way, any bias
incurred by other authors through their experience
of appraisals in the PCT was minimised.
RESULTS
Demographics
There were a total number of 107 GPs appraised
by the PCT. The breakdown of numbers by type of
role and age is provided in Table 1a and b
respectively. The division between age groups of
GPs has been found to be a useful tool in the
development of GP workforce planning considera-
tions.19
The response rate was 60% (n=64).20
Appraisees’ feelings prior to the
appraisal
The results show that the appraisee’s confidence
and feeling of comfort with the process increased
with exposure to it. Positive comments (Table 2)
were received from respondents who had already
experienced the appraisal process at least once.
Experience of exposure to appraisals might
account for the high number of positive responses
towards the impending appraisal (66%). One first
timer felt that he/she was ‘. . . doubtful of the
value of appraisals especially if associated with
revalidation’. A third timer reflected that he/she
felt ‘. . . not as apprehensive as previously. At last
beginning to understand the process’. The com-
ments collected from respondents who were
familiar with the process show evidence who
appraisals were perceived as being ‘. . . motiva-
tional . . .’, ‘. . . beneficial . . .’, about ‘. . . achieve-
ment . . .’ and about what one is going . . .‘to do
next’ even before they take place.
Quality of feedback received from the
appraiser
All 64 respondents (Table 2) felt that the apprais-
ers were well prepared, knowledgeable about the
process, provided them with a chance to say
everything they wanted to say, had a good
rapport with the appraisee, and had a good end
discussion. The majority of the appraisees (99%;
Table 2) felt that the appraiser supported them to
think about their developmental needs. This is in
spite of the 25% who had previously felt slightly
apprehensive about the appraisal. Many of the
appraisals took longer than 2 hours (67%).
Perceived impact of the appraisal
The majority of appraisees felt that the process
helped them identify clear learning aims for the
next year (84%; Table 3). Only 2% felt that these
could have been clearer. Appraisees also felt that
the learning aims identified would be easily
achieved within the year (58%), and only 7% felt
that they would not be able to achieve these with
ease. In spite of the challenges imposed by the
learning aims, all appraisees felt that these would
be completed to some extent within the year. In
fact, 82% were somewhat confident in achieving
them. Fifty percent of the appraisees felt that the
appraisal had contributed to a reasonable feed-
back on their work as a GP while some felt that it
had contributed either not at all or only slightly
(14%). Appraisees felt that the appraisal had
helped them to build up confidence as GPs (46%),
and 17% felt that the appraisal had not helped in
building their confidence. More importantly, some
aprraisees felt valued as GPs (58%) as opposed
to 19% where the appraisal had no effect on their
perceived value as GPs. The appraisees also felt
that the appraisal would have an effect on improv-
ing patient care (35%). About a third of the
respondents felt that the appraisal would either
have no effect at all or little effect (29%). Interest-
ingly, 8% of the 37 respondents felt that their
appraisal would in fact contribute to a delay in
retirement age.
Potential learning outcomes from Form
4s
The categories of learning outcomes identified
varied from clinical skills and knowledge and
being an effective part of the practice to running a
practice and personal, professional life coaching
needs (Table 2). The most frequent clinical learn-
ing needs identified were: carrying out an audit
(28%), recording PUNS and DENS` (22%) and
keeping up to date with clinical skills and knowl-
edge (14%). Learning outcomes that relate to the
running of the practice included: learning about
clinical systems (22%), general IT learning (19%)
and having practice-based learning in protected
Table 1a Breakdown of GP numbers by type
GP type Number of GPs
GP principals 80
Locum doctors 17
GP retainers 4
Flexible careers scheme 3
Primary Medical Services salaried GPs 3
Table 1b Breakdown of GP numbers by age
GP age groups Percentage of GPs
Under 34 16
34–49 years 52
50+ years 32
The formative impact of general practice appraisals 447
time (11%). The personal or professional life
coaching outcomes included retirement planning
(8%), finding the right practice (8%) and taking a
sabbatical (6%).
Qualitative analysis from comments
about the appraisal
This part of the questionnaire was largely left
unanswered by respondents. Those who
responded felt positive about the influence that
the appraisal could have on them. There were
also some suggestions for improvement on the
overall appraisal process (Box 1). The perceived
educational value of the appraisal is illustrated by
the comments collected in Box 1: ‘. . . advised me
on how to achieve them (learning needs)’, ‘. . . it
helped me focus . . . how my career is progres-
sing’. Some suggest a potential impact of apprai-
sals on the medical community as a whole: ‘. . .
rekindled my enthusiasm to do more teaching
work . . .’; ‘. . . useful career advice . . .’. Some
respondents are therefore thinking broadly about
their work as GPs and about their personal indivi-
dual strengths.
DISCUSSION
We present findings on the impact of appraisals
on appraisees using the purely formative model
for appraisals. The appraisees in this study show
an overall satisfaction with the appraisal model
used here and provide evidence for anticipated
appraisal impact. Appraisees use words such as
‘inspirational’, ‘motivational’, ‘rekindled my enthu-
siasm’, ‘encouraging’ to express the motivational
role behind appraisals that is often essential for
Table 2 Appraisees’ feelings before the appraisal and the quality of the appraiser’s feedback
Before the appraisal
Positive (%) Apprehensive (%) Negative (%) Neutral (%)
Overall, how did you feel about your 66 25 1 8
impending appraisal?
Selection of positive comments
‘. . .not as apprehensive as previously. At last beginning to understand the process.’
‘I felt much more comfortable. . .’
‘. . . knew what to expect. . .’
‘. . . good to spend some time on oneself.’
‘They always fire me up with some enthusiasm and motivation.’
‘. . . I enjoy putting together what I have achieved rather what I didn’t get. . .’
. . .’ great. . .to think about what next!!!’
‘. . . I look forward to it.’
‘The appraiser is a GP guru.’
‘Appraisals have always been a beneficial process.’
Selection of less positive comments
‘I felt I had failed to collect enough evidence’
‘slightly apprehensive. . .’ need to pass’’
‘I had time off to prepare. . .’
‘. . . I felt that an in depth reflection of my practise was going to have less relevance. . .’
‘I am doubtful of the value of appraisals especially if associated with revalidation.’
Quality of the feedback provided by the appraiser
Number of respondents out of 64
During the appraisal Yes No
Was the appraiser well prepared? 64 0
Was the appraiser knowledgeable about the appraisal process? 64 0
Did you have the chance to say all that you wanted to say? 64 0
Did you feel that you had a good rapport with your appraiser? 64 0
At the end, did the appraiser bring your discussion to a positive conclusion? 64 0
I felt that the appraiser supported me to think about my own development needs and
how I can meet them 99%
I felt that I was being assessed by the appraiser 1%
Would you wish to be appraised by this appraiser again?
Yes 99% Unsure 1% No 0%
448 O Zolle, M Rickenbach and E Gorrod
any personal developmental experience. The fact
that the learning needs that have been identified
this year will be reviewed the next also acts as a
catalyst to engage in further professional learning
activities.
The success of the formative appraisal there-
fore lies in the discussion between appraiser and
appraisee about the individual educational out-
comes that may be achieved over the year. It is
therefore important in the formative model that
appraisees feel they are ‘moving forward’ in their
work and careers. Appraisees report in this paper
that this form of appraisal could have an impor-
tant beneficial impact on patient care. It is the
feeling of ‘being valued as a GP’ that is important
to many appraisees. We show that the learning
outcomes achieved through the formative model
of appraisals range from the clinical to the life
coaching ones. It is, therefore, not surprising that
experienced appraisees look forward to their
appraisal as they emphasise each individual’s
personal strengths, thereby being a characteristic
of the formative model of appraisals. The litera-
ture on appraisals is flourishing with reports
about adequate appraiser training as appraisals
depend on the model of appraisal followed as
well as the knowledge and training received by
the appraiser. It is interesting to note that
appraisers using a combination of summative and
formative processes as part of the revalidation
model have claimed this to be an inadequate pro-
cess that is disengaged from the role of a trainer
or educator.21
The model for appraisals that has been sug-
gested for England proposes the inclusion and
approval of evidence with explicit judgements
made about the appraisees’ performance.22
In
Scotland, however, a more formative model is fol-
lowed based around the five core categories iden-
tified for appraisals, allowing the discussion of one
category in depth at each appraisal over five years
and revalidation is incorporated through participa-
tion in an annual developmental appraisal. In the
latter model appraisees have been found to derive
value and benefit from the formative aspect of the
appraisal as they were able to link these to
improved practice.23
It has therefore been sug-
gested that revalidation and appraisal remain
separate.23
In Wales the appraisal process is
tightly linked to the delivery of a quality-assured
systematic process that manages virtual appraisal
folders, selection of appraisers and booking
system for appraisal discussions, and manages
follow-up actions with the development of personal
Table 3 The impact of the appraisal; how the appraisees felt that the appraisal had contributed to their overall educational de-
velopment within the areas specified.
Perceived appraisal impact by trainees
How much do you feel the appraisal has contributed to; Not at 2 (%) 3 (%)4 (%) As much as Number of
all possible responses
5 (%) (total of 64
respondents)
1. Clarity of learning aims for the next year. 0 2 14 43 41 44
2. Ease of achieving those aims 7 2 33 46 12 43
3. Likelihood of completing your learning aims for this year 0 3 15 62 20 40
4. Feedback received on your work as a GP. 5 9 36 25 25 44
5. Confidence as a GP 5 12 37 30 16 43
6. How valued do you feel as a GP. 5 14 23 35 23 43
7. Improved patient care 12 17 36 21 14 42
8. A delay in retirement age 68 5 19 3 5 37
Top three more frequent learning outcomes
Clinical (on being a doctor) Carrying out an Recording PUNS Keeping up
audit & DENS to date
Frequency score out of 64 18 14 9
Percentage frequency (%) 28 22 14
Running the practice (on being a businessman)
Clinical systems IT learning
Practice based
learning
protected time
Frequency score out of 64 14 12 7
Percentage frequency (%) 22 19 11
Personal (on being a human being)
Retirement planning Finding the right Taking a
practice sabbatical
Frequency score out of 64 5 5 4
Percentage frequency (%) 8 8 6
The formative impact of general practice appraisals 449
development plans.24
The current process in Wales
is formative but there is a debate about how this
could be linked to a more summative form of
appraisals for revalidation. Overall, the emphasis
in the literature so far has been on the provision of
adequate training for appraisers, who are not paid
any extra income to carry out their roles.
It has been reported that the training of peer
appraisers is crucial to the effectiveness of the
appraisal process.3,7,21
The success of appraisals
is partly determined by the quality of the feedback
given by a trained professional peer.3,7,21
The role
of the appraiser as a source for invaluable specia-
lised information is fundamental to the potential
impact of the appraisal process. The training that
the appraisers received appears to elicit some
important positive outcomes from the experience.
The majority of appraisees in our study (99%) felt
that the appraiser supported them to think about
their own developmental needs and how to meet
them. Appraisees also value the chance that
appraisals offer for discussing issues pertaining
to the practice and to their overall careers with a
knowledgeable peer. Furthermore, it has been
shown that there are inaccuracies found in the
assessment of one’s own learning needs and that
the identification of gaps in one’s own overall per-
formance is not always translated into an effective
change in behaviour.23
Well-trained appraisers should contribute to the
following developmental challenges experienced
by appraisees as identified by Jenkins (2003).22
Those who:
. wish to develop but have limited experience of the
educational methods for doing so
. have limited experience of identifying learning
needs
Box 1 Positive and Less positive comments from Appraisees regarding the educational and developmental
value of the appraisal.
Positive educational and developmental comments
‘Made me think about my learning needs and concrete them in written form. . .making them more likely to be
achieved.’
‘I feel more positive about my educational needs.’
‘. . .identified strengths and weaknesses . . ..’
‘. . .good in elucidating important areas to include in my portfolio.’
‘. . .it helped me focus on my clinical and learning needs and how my career is progressing.’
‘. . .helped prepare my learning needs with ease.’
‘. . .encouraged me to obtain objectives . . .and think about what I’m learning and achieving.’
‘. . .guided me through identifying my learning needs/advised me on how to achieve them.’
‘Useful suggestions for how to achieve my learning outcomes, useful career advice.’
‘. . .rekindled my enthusiasm to do more teaching work.’
‘. . .constructive approach to personal development. . .inspiration to get on’
‘Good opportunity to reflect on my current role. . .’
‘. . .realistic and productive. . .’
‘Opportunity to reflect on achievements and positive aspects. . .in the workplace’
‘. . .cover all areas of development, interest and concern. . .’
‘Clear thoughtful reflection of my documentation; challenging and supportive.’
‘Encouraging.’
Total positive comments: 48 out of 64 (75%)
Sample of positive educational or developmental comments about the GP in the practice
‘. . .making a realistic plan of how to address the aims and challenges of the practice.’
‘Good to discuss with a neutral person outside the practice.’
‘Good. . .to talk to an experienced objective and understanding peer, in confidence.’
‘Opportunity for confidential discussion about my practices and changes.’
‘Space to think.’
‘Chance to speak about my concerns and the stresses I have been under over the past year.’
Less positive issues
‘Using the online toolkit.’
‘. . .time taken to complete form 4 and then cross examination to the PDP; are both really required?’
‘I could have been challenged a bit more.’
‘. . .not really helpful to develop my practice.’
‘I hate appraisals as my mentality makes me concentrate on the negative. . .’
‘’. . .some repetition. . .’
‘. . .get a report from the practitioners in my practice.’
‘. . .can be repetitive. . .’
‘. . .process contains elements of jumping through hoops to prove I do the things I already know I do.’
‘. . .I remain unconvinced about the whole appraisal process. It is bureaucratic and I do not think generally
helpful. A lot of form filling. . .little added self reflection.’
‘I think mentoring would be more appropriate.’
Total less positive comments: 15 out of 64 (23%)
450 O Zolle, M Rickenbach and E Gorrod
. have difficulty in separating learning aims from
objectives
. find it difficult to recognise the usefulness of com-
plaints for exploring learning needs
. understand how to identify learning needs but dis-
like recording the information
. find it difficult to achieve or prioritise all their
learning objectives
. find it difficult to commit themselves to areas of
learning and self-development.
The uniqueness of appraisals lies precisely in the
ability to provide professional guidance in all of
the above areas. No other current form of profes-
sional development or formative assessment for
GPs provides for this directly tailored to the indi-
vidual. Learning objectives should be achievable
and realistic but they should also pose an educa-
tional and/or a developmental challenge.
Our findings on the feeling of confusion related
to the nature of appraisals and their overall edu-
cational value are in agreement with the litera-
ture.3,5,16
Limitations to the study
. We have no control groups for this study and no
assessment of the non-responder group.
. We may only report the potential impact of
appraisals as perceived by the appraisees and as
revealed from their learning outcomes.
. We cannot assume that the majority of the apprais-
ees will have achieved their learning goals.
. It is also possible that the appraisee might have
been favourably biased in their response as
affected by the relationship and time spent with
the appraiser.
Implications for future research and
clinical practice
The GMC has proposed a system that will
encompass both appraisal and assessment for
revalidation but if there were to be a combined
system there would be a need to ensure that the
professional development focus and the educa-
tional impact brought in by the current appraisal
system are not lost. McKinstry et al (2005)3
have
shown that appraisees are concerned with the
current links that are being made with revalida-
tion. These concerns stem from the potential for
clashes between the appraiser and the appraisee
if the system for revalidation were to be imposed
as part of the appraisal process. Appraisals are
not about poor performance, they are about
improving the performance of all and for the ben-
efit of both the public and the medical commu-
nity.
Funding
This study received no funding. The researcher
and main writer for this study, OZ, is employed by
NHS Education South Central. The Wessex Dean-
ery is part of NHS Education South Central.
Ethical approval
None required. All the data received from partici-
pants were anonymised from the source.
Conflicts of interest
None.
References
1 West MA, Borrill C and Dawson J (2006) The link
between the management of employees and patient
mortality in acute hospitals. International Journal of
Human Resources Management 13: 1299–310.
2 Adams R, Iling J, Jelley D, Walker C and van Zwanen-
berg T (2005) The critical success factors in delivering
GP appraisal: a review of the first year in Northumber-
land and Tyne and Wear; Sunderland Teaching PCT.
Postgraduate Institute for Medicine and Dentistry: Sun-
derland.
3 McKinstry B, Peacock H and Shaw J (2005) Experiences
of partner and external peer appraisal. British Journal
of General Practice 55: 539–43.
4 Boylan O, Bradley T and McKnight A (2005) GP percep-
tions of appraisal: professional development, perfor-
mance management, or both? British Journal of General
Practice 55(516): 544–5.
5 Middlemas J and Siriwardena AN (2003) General practi-
tioners, revalidation and appraisal: a cross sectional
survey of attitudes, beliefs and concerns in Lincolnshire.
Medical Education 37: 778–85.
6 Chambers R, Tavabie A and See S (2003) Exploring
ideas for the evaluation of the GP appraisal system.
Education for Primary Care 14: 579–83.
7 Lewis M, Elwyn G and Wood F (2003) Appraisal of family
doctors: an evaluation study. British Journal of General
Practice 53: 454–60.
8 Dhaliwal JS (2003) Why appraisal is not performance
management – or is it? Education for Primary Care 14:
555–7.
9 Conlon M (2003) Appraisal: the catalyst of personal
development. BMJ 327: 389–91.
10 General Medical Council (2001) Good Medical Practice.
GMC: London.
11 Martin D, Harrison P, Joesbury H and Wilson R (2006)
Appraisal for GPs. School of Health and Related
Research, University of Sheffield.
12 Department of Health. CMO Update, Issue 44.
www.dh.gov.uk/cmo
13 Sharpe B (2003) Revalidation and the quality agenda.
Education for Primary Care 14: 543–7.
14 Thomas P (2005) Will GP revalidation make things
better or worse? British Journal of General Practice
318–19.
15 Mohanna K (2003) Appraisal: what now? Education for
Primary Care 14: 537–9.
16 Torjesen I (2006) Moral dilemma for rural GPs. Pulse, 16
November.
17 Bruce D, Phillips K, Reid R, Snadden D and Harden R
The formative impact of general practice appraisals 451
(2004) Revalidation for general practitioners: rando-
mised comparison of two revalidation models. BMJ 328:
687–91.
18 Gorrod E (2006) Annual Report on GP Appraisal in East-
leigh and TVS PCT 2005–2006. Eastleigh and Test Valley
South PCT: Hampshire.
19 Smith F, Scallan S, Bell J and Curson J (2006) Training
for General Practice in Hampshire & the Isle of Wight:
2006–2010. A Severn and Wessex Deanery Report.
20 Oppenheim AN (1992) Questionnaire Design, Interview-
ing and Attitude Measurement. Continuum International
Publishing: London.
21 Baron R, Agius S and Lewis B (2007) GP appraisal: per-
ceptions of appraisers and appraisees. Education for
General Practice 18: 373–81.
22 Jenkins M (2003) Guide to utter failure: a tool for provid-
ing insight into the appraisal process. Education for Pri-
mary Care 14: 569–71.
23 Wakeling J, Cameron N and Lough M (2008) What factors
impact on general practitioners’ engagement with
appraisal? A qualitative evaluation in Scotland. Educa-
tion for Primary Care 19: 615–23.
24 Rhydderch M, Langharne K, Manfell N, Powncwby P and
Lewis M (2008) Developing a skills based model to pro-
mote effective appraisal discussions amongst GPs in
Wales. Education for Primary Care 19: 496–505.
Correspondence to: Dr Olga Zolle, Romsey Road, High-
croft, Winchester SO22 5DH, UK. Email: Olga.Zolle@
nesc.nhs.uk
Accepted October 2009
452 O Zolle, M Rickenbach and E Gorrod
Appendix
APPRAISEE FEEDBACK FORM
Appraisee: Appraiser:
Before the appraisal . . .
Overall, how did you feel about your impending appraisal?
Positive Apprehensive Negative Neutral
Please add comments if desired:
During the appraisal . . .
Was the appraiser well prepared?
Yes No
Was the appraiser knowledgeable about the appraisal process?
Yes No
Did you have the chance to say all that you wanted to say?
Yes No
Did you feel that you had a good rapport with your appraiser?
Yes No
At the end, did the appraiser bring your discussion to a positive conclusion?
Yes No
How long did your appraisal meeting last?
Less than 1 hour 1–2 hours More than 2 hours
Overall, which of these two statements more closely reflects your experience of this interview?
I felt that the appraiser supported me to think I felt that I was being assessed by the
about my own development needs and how appraiser
I can meet them
Would you wish to be appraised by this appraiser again?
Yes No
If you have answered ‘No’ to any of the questions in this section, please explain why you feel this:
The formative impact of general practice appraisals 453
Since you have been undergoing appraisal . . .
On a scale of 1 to 5, where 1 = not at all and 5 = as much as possible, please rate how much do you
feel appraisal has contributed to:
a) How valued you feel as a general practitioner
b) The feedback you receive on your work as a general practitioner
c) Your confidence as a general practitioner
d) The clarity of your learning aims for the next year
e) The ease of achieving those aims
f) The likelihood of you completing your learning aims for this year
g) A delay in your retirement age
h) Improved patient care
General . . .
What was good about your appraisal?
What aspects did you find unproductive; could your appraiser have done anything differently to help?
454 O Zolle, M Rickenbach and E Gorrod

More Related Content

What's hot

Evidence based practice by Tanoj Patidar MSc Nursing (MSN)
Evidence based practice by Tanoj Patidar MSc Nursing (MSN)Evidence based practice by Tanoj Patidar MSc Nursing (MSN)
Evidence based practice by Tanoj Patidar MSc Nursing (MSN)Tanoj Patidar
 
Evidence Base Practice (EBP)-Define, Benefits,Resource, steps PPT
Evidence Base Practice (EBP)-Define, Benefits,Resource, steps PPTEvidence Base Practice (EBP)-Define, Benefits,Resource, steps PPT
Evidence Base Practice (EBP)-Define, Benefits,Resource, steps PPTsonal patel
 
Evidence Based Practice
Evidence Based PracticeEvidence Based Practice
Evidence Based PracticeSreeraj S R
 
Evidence based practice
Evidence based practiceEvidence based practice
Evidence based practiceVini Mehta
 
Evidence-Based Practice and the Future of Nursing
Evidence-Based Practice and the Future of NursingEvidence-Based Practice and the Future of Nursing
Evidence-Based Practice and the Future of NursingOther Mother
 
evidence based practice, EBP
evidence based practice, EBPevidence based practice, EBP
evidence based practice, EBPpankaj rana
 
Doctorate Dissertation Proposal
Doctorate Dissertation ProposalDoctorate Dissertation Proposal
Doctorate Dissertation ProposalMaurice Dawson
 
Information interventions for injury recovery: a review
Information interventions for injury recovery: a reviewInformation interventions for injury recovery: a review
Information interventions for injury recovery: a reviewAlex Collie
 
2015_Discrete choice experiments versus rating scale exercises to evaluate th...
2015_Discrete choice experiments versus rating scale exercises to evaluate th...2015_Discrete choice experiments versus rating scale exercises to evaluate th...
2015_Discrete choice experiments versus rating scale exercises to evaluate th...Cindy Noben
 
Assessing health status & health needs
Assessing health status & health needsAssessing health status & health needs
Assessing health status & health needsMukace Karn
 
objective structured clinical examination
objective structured clinical examinationobjective structured clinical examination
objective structured clinical examinationBhagwat Raut
 
Health Need Assessment
Health Need AssessmentHealth Need Assessment
Health Need AssessmentRajbir Kaur
 
Introduction to Evidence-Based Practice
Introduction to Evidence-Based PracticeIntroduction to Evidence-Based Practice
Introduction to Evidence-Based Practicelulurosalesrnbsn
 
Living evidence 3
Living evidence 3Living evidence 3
Living evidence 3stanbridge
 
Developing implementing patient reported outcome measures for dietetics pr...
Developing  implementing patient reported outcome measures for dietetics   pr...Developing  implementing patient reported outcome measures for dietetics   pr...
Developing implementing patient reported outcome measures for dietetics pr...Keith Meadows
 

What's hot (20)

Evidence based practice by Tanoj Patidar MSc Nursing (MSN)
Evidence based practice by Tanoj Patidar MSc Nursing (MSN)Evidence based practice by Tanoj Patidar MSc Nursing (MSN)
Evidence based practice by Tanoj Patidar MSc Nursing (MSN)
 
Evidence Base Practice (EBP)-Define, Benefits,Resource, steps PPT
Evidence Base Practice (EBP)-Define, Benefits,Resource, steps PPTEvidence Base Practice (EBP)-Define, Benefits,Resource, steps PPT
Evidence Base Practice (EBP)-Define, Benefits,Resource, steps PPT
 
Evidence Based Practice
Evidence Based PracticeEvidence Based Practice
Evidence Based Practice
 
Evidence based practice
Evidence based practiceEvidence based practice
Evidence based practice
 
Evidence-Based Practice and the Future of Nursing
Evidence-Based Practice and the Future of NursingEvidence-Based Practice and the Future of Nursing
Evidence-Based Practice and the Future of Nursing
 
evidence based practice, EBP
evidence based practice, EBPevidence based practice, EBP
evidence based practice, EBP
 
Doctorate Dissertation Proposal
Doctorate Dissertation ProposalDoctorate Dissertation Proposal
Doctorate Dissertation Proposal
 
Introduction to EBP
Introduction to EBPIntroduction to EBP
Introduction to EBP
 
Nursing process
Nursing processNursing process
Nursing process
 
Information interventions for injury recovery: a review
Information interventions for injury recovery: a reviewInformation interventions for injury recovery: a review
Information interventions for injury recovery: a review
 
2015_Discrete choice experiments versus rating scale exercises to evaluate th...
2015_Discrete choice experiments versus rating scale exercises to evaluate th...2015_Discrete choice experiments versus rating scale exercises to evaluate th...
2015_Discrete choice experiments versus rating scale exercises to evaluate th...
 
Assessing health status & health needs
Assessing health status & health needsAssessing health status & health needs
Assessing health status & health needs
 
Dr V K Tiwari
Dr V K TiwariDr V K Tiwari
Dr V K Tiwari
 
objective structured clinical examination
objective structured clinical examinationobjective structured clinical examination
objective structured clinical examination
 
Health Need Assessment
Health Need AssessmentHealth Need Assessment
Health Need Assessment
 
Introduction to Evidence-Based Practice
Introduction to Evidence-Based PracticeIntroduction to Evidence-Based Practice
Introduction to Evidence-Based Practice
 
Living evidence 3
Living evidence 3Living evidence 3
Living evidence 3
 
Planning
PlanningPlanning
Planning
 
Developing implementing patient reported outcome measures for dietetics pr...
Developing  implementing patient reported outcome measures for dietetics   pr...Developing  implementing patient reported outcome measures for dietetics   pr...
Developing implementing patient reported outcome measures for dietetics pr...
 
Myths and reality in evidence based nursing
Myths and reality in evidence based nursingMyths and reality in evidence based nursing
Myths and reality in evidence based nursing
 

Similar to Appraisal Paper Educ Prim Care 2010 pp445-54

Methods Of Program Evaluation. Evaluation Research Is Offered
Methods Of Program Evaluation. Evaluation Research Is OfferedMethods Of Program Evaluation. Evaluation Research Is Offered
Methods Of Program Evaluation. Evaluation Research Is OfferedJennifer Wood
 
Reliability And Validity
Reliability And ValidityReliability And Validity
Reliability And ValidityCrystal Torres
 
ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...
ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...
ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...Gina Rizzo
 
1CPD researchStudent nameInstructor nameCourse.docx
1CPD researchStudent nameInstructor nameCourse.docx1CPD researchStudent nameInstructor nameCourse.docx
1CPD researchStudent nameInstructor nameCourse.docxfelicidaddinwoodie
 
There needs to be a seperate response to each peers posting and it .docx
There needs to be a seperate response to each peers posting and it .docxThere needs to be a seperate response to each peers posting and it .docx
There needs to be a seperate response to each peers posting and it .docxOllieShoresna
 
Introduction of Objective Structured Clinical Examination as assessment tool ...
Introduction of Objective Structured Clinical Examination as assessment tool ...Introduction of Objective Structured Clinical Examination as assessment tool ...
Introduction of Objective Structured Clinical Examination as assessment tool ...iosrjce
 
Evaluating the quality of quality improvement training in healthcare
Evaluating the quality of quality improvement training in healthcareEvaluating the quality of quality improvement training in healthcare
Evaluating the quality of quality improvement training in healthcareDaniel McLinden
 
ACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docx
ACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docxACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docx
ACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docxbobbywlane695641
 
2014 CGSA HolisticReviewPoster
2014 CGSA HolisticReviewPoster2014 CGSA HolisticReviewPoster
2014 CGSA HolisticReviewPosterAlison Martin
 
CGEA2011Patient Centeredness
CGEA2011Patient CenterednessCGEA2011Patient Centeredness
CGEA2011Patient CenterednessAlison Martin
 
Chapter 5 Program Evaluation and Research TechniquesCharlene R. .docx
Chapter 5 Program Evaluation and Research TechniquesCharlene R. .docxChapter 5 Program Evaluation and Research TechniquesCharlene R. .docx
Chapter 5 Program Evaluation and Research TechniquesCharlene R. .docxchristinemaritza
 
Respond to the post bellow, using one or more of the followi
Respond to the post bellow, using one or more of the followiRespond to the post bellow, using one or more of the followi
Respond to the post bellow, using one or more of the followimickietanger
 
Nursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docx
Nursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docxNursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docx
Nursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docxcherishwinsland
 
Assignment DescriptionA reputable hospital has high quality .docx
Assignment DescriptionA reputable hospital has high quality .docxAssignment DescriptionA reputable hospital has high quality .docx
Assignment DescriptionA reputable hospital has high quality .docxluearsome
 
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdfWare DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdfJoseErnestoBejarCent
 
Implementing SBARStudents NameInstitutional
Implementing SBARStudents NameInstitutionalImplementing SBARStudents NameInstitutional
Implementing SBARStudents NameInstitutionalMalikPinckney86
 
Exploring the Economics of Quality Improvement Education in Healthcare: An A...
Exploring the Economics of Quality Improvement Education in Healthcare:  An A...Exploring the Economics of Quality Improvement Education in Healthcare:  An A...
Exploring the Economics of Quality Improvement Education in Healthcare: An A...Daniel McLinden
 
Assessment of healthcare providers’ collaboration at governmental hospitals
Assessment of healthcare providers’ collaboration at governmental hospitalsAssessment of healthcare providers’ collaboration at governmental hospitals
Assessment of healthcare providers’ collaboration at governmental hospitalsiyad shaqura
 
Sample employee appraisal comments
Sample employee appraisal commentsSample employee appraisal comments
Sample employee appraisal commentsaasifmandvi595
 

Similar to Appraisal Paper Educ Prim Care 2010 pp445-54 (20)

Methods Of Program Evaluation. Evaluation Research Is Offered
Methods Of Program Evaluation. Evaluation Research Is OfferedMethods Of Program Evaluation. Evaluation Research Is Offered
Methods Of Program Evaluation. Evaluation Research Is Offered
 
Reliability And Validity
Reliability And ValidityReliability And Validity
Reliability And Validity
 
ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...
ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...
ARCADO-Adding Random Case Analysis To Direct Observation In Workplace-Based F...
 
1CPD researchStudent nameInstructor nameCourse.docx
1CPD researchStudent nameInstructor nameCourse.docx1CPD researchStudent nameInstructor nameCourse.docx
1CPD researchStudent nameInstructor nameCourse.docx
 
There needs to be a seperate response to each peers posting and it .docx
There needs to be a seperate response to each peers posting and it .docxThere needs to be a seperate response to each peers posting and it .docx
There needs to be a seperate response to each peers posting and it .docx
 
Summative Essay
Summative EssaySummative Essay
Summative Essay
 
Introduction of Objective Structured Clinical Examination as assessment tool ...
Introduction of Objective Structured Clinical Examination as assessment tool ...Introduction of Objective Structured Clinical Examination as assessment tool ...
Introduction of Objective Structured Clinical Examination as assessment tool ...
 
Evaluating the quality of quality improvement training in healthcare
Evaluating the quality of quality improvement training in healthcareEvaluating the quality of quality improvement training in healthcare
Evaluating the quality of quality improvement training in healthcare
 
ACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docx
ACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docxACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docx
ACT500 Research Evaluation TablesArticle 1 Measuring Perfo.docx
 
2014 CGSA HolisticReviewPoster
2014 CGSA HolisticReviewPoster2014 CGSA HolisticReviewPoster
2014 CGSA HolisticReviewPoster
 
CGEA2011Patient Centeredness
CGEA2011Patient CenterednessCGEA2011Patient Centeredness
CGEA2011Patient Centeredness
 
Chapter 5 Program Evaluation and Research TechniquesCharlene R. .docx
Chapter 5 Program Evaluation and Research TechniquesCharlene R. .docxChapter 5 Program Evaluation and Research TechniquesCharlene R. .docx
Chapter 5 Program Evaluation and Research TechniquesCharlene R. .docx
 
Respond to the post bellow, using one or more of the followi
Respond to the post bellow, using one or more of the followiRespond to the post bellow, using one or more of the followi
Respond to the post bellow, using one or more of the followi
 
Nursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docx
Nursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docxNursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docx
Nursing and Health Sciences (2005), 7, 281 – 283© 2005 Bla.docx
 
Assignment DescriptionA reputable hospital has high quality .docx
Assignment DescriptionA reputable hospital has high quality .docxAssignment DescriptionA reputable hospital has high quality .docx
Assignment DescriptionA reputable hospital has high quality .docx
 
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdfWare DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
Ware DEFINING AND MEASURING PATIENT SATISFACTION 1983.pdf
 
Implementing SBARStudents NameInstitutional
Implementing SBARStudents NameInstitutionalImplementing SBARStudents NameInstitutional
Implementing SBARStudents NameInstitutional
 
Exploring the Economics of Quality Improvement Education in Healthcare: An A...
Exploring the Economics of Quality Improvement Education in Healthcare:  An A...Exploring the Economics of Quality Improvement Education in Healthcare:  An A...
Exploring the Economics of Quality Improvement Education in Healthcare: An A...
 
Assessment of healthcare providers’ collaboration at governmental hospitals
Assessment of healthcare providers’ collaboration at governmental hospitalsAssessment of healthcare providers’ collaboration at governmental hospitals
Assessment of healthcare providers’ collaboration at governmental hospitals
 
Sample employee appraisal comments
Sample employee appraisal commentsSample employee appraisal comments
Sample employee appraisal comments
 

Appraisal Paper Educ Prim Care 2010 pp445-54

  • 1. The formative impact of general practice appraisals: a post-appraisal questionnaire survey Olga Zolle BSc PhD PGCE Research and Development Manager, NHS Education South Central, Winchester, UK Mark Rickenbach FRCP FRCGP DCH FHEA PhD Associate Dean and GP Trainer, NHS Education South Central, Wessex Deanery and Honorary Fellow, University of Winchester, Winchester, UK Eileen Gorrod BM FRCGP GP Appraisal Clinical Lead NHS Education South Central, Winchester, UK SUMMARY Since their national implementation in 2002, GP appraisals are being extensively applied. There is a positive link between appraisals and patient care in hospitals.1 There also appears to be no uniform model for appraisals,2 which has led to confusion between their ‘stand-alone’ role as a formative assessment tool3–5 and their contribu- tion to summative revalidation. This is a case study into the formative ele- ment of appraisals as carried out at a primary care trust in Wessex. We offer an insight into the diversity of learning outcomes and develop- mental potential of the formative element of appraisals (excluding revalidation) attained through the analysis of emerging themes from responses to a purposely designed question- naire aimed at appraisees (questions appear in the tables). To our knowledge this is one of the few research articles that deal with the formative edu- cational element of GP appraisals. INTRODUCTION Appraisals were originally designed to act as developmental and hence formative tools for assessment and not as performance management tools.3,6–,8 They provide opportunities for confiden- tial discussion of the problems and solutions Education for Primary Care (2009) 20: 445–54 # 2009 Radcliffe Publishing Limited WHAT IS ALREADY KNOWN IN THIS AREA . The formative element of appraisals provides a unique opportunity for discussion of learning needs and outcomes with a trained peer. . The extent of the impact of this form of appraisal remains unclear. WHAT THIS WORK ADDS . Main impacts of appraisals identified by appraisees are an increase in their confidence, feeling valued in their jobs, improvement in overall patient care and contributing to a delay in retirement age. . It emphasises the need for purpose-trained appraisers. The main clear, challenging and achievable learning objectives analysed as a result of the study were in clinical skills, running of a practice and life coaching, suggesting a need to train appraisers to provide advice in these areas. SUGGESTIONS FOR FUTURE RESEARCH . Comparison of learning outcomes arising from appraisals with and without revalidation. . Provide further insight into the training of appraisers, particularly on the formative element of the appraisal through an in-depth analysis of the types of learning outcomes that might stem from it. Keywords: appraisal, appraisal model, formative assessment, peer assessment, revalidation
  • 2. encountered in general practice and are intended to help identify appropriate challenging learning needs that are translated into outcomes. They may therefore influence and improve standards in a non-judgemental manner.3,7 Research shows that GP appraisals act as a medium for discussion of issues arising from everyday practice and are per- ceived as beneficial in reflecting individual oppor- tunities and overall practice performance.3,7,9 The aims of appraisals are in contrast with the General Medical Council’s (GMC’s) revalidation system of summative assessments that consider whether an individual is fit for practice.6,10 Revalid- ation has imposed outcomes and is a matter of public record.11,12 Appraisal is being put forward as an indicator for revalidation, the application of clinical governance systems and also to track Good Medical Practice.13 However, Middlemas and Siriwardena (2003)5 performed a cross-sec- tional study amongst GPs and concluded that appraisals ‘might not detect incompetent doctors’. The problem is how these two very distinct sys- tems (one is confidential and formative while the other is in the public domain and summative) can be brought together in an effective manner with- out disturbing the unique nature, effectiveness and positive educational impact that appraisals may provide for GP practice.14 Another argument for joining appraisals with revalidation is cost effectiveness:15,16 Bruce et al (2004)17 compared two distinct methods for revalidation, one linked to appraisals and one that is independent. They report that ‘peer or educational support will be required’ for both methods. METHODS At the time of the study the PCT is working towards the standards set in Assuring the Quality of Medical Appraisal by the NHS Clinical Govern- ance Support Group – July 2005.18 In contrast to other appraisal models currently maintained across the country, this PCT uses a model for appraisal set clearly apart from revalidation. Although the personal development plan is revis- ited during the appraisal this is not considered to be a formal shared outcome. All the appraisers at this PCT attend: . regular developmental meetings . a developmental day . other non-compulsory conferences and events related to their training for example the National Association of Primary Care Educators’ appraisal conference held annually. The developmental day takes place once every 18 months. The developmental meetings last two hours and occur three times a year. The meetings are attended by a GP tutor who is able to provide an overview of the appraisal process across the strategic health authority (SHA) and nationally. These meetings act as a forum for the exchange of ‘best practice’ ideas, updating and improve- ment of the process. At the time of writing, speci- fic emphasis is being placed on the issue of ‘moving the appraisee on’. For the purpose of this study, all the GPs who were appraised over the period of a year (from April 2005 to April 2006) were asked to complete a questionnaire regarding the educational impact and quality of the appraisal. The appraisees selected the appraiser from a list provided by the PCT. The appraiser then contacted the appraisee to confirm the date and time of appraisal as well as providing an outline of the process. The apprai- sal process ensured that the personal develop- ment plan followed the SMART criteria of specifi- city, measurability, achievability, relevance and time bound. The confidential paperwork needed to provide support and evidence for the process was available to the appraiser any time over two weeks prior to the appraisal. Half a day was allowed for the appraisal interview and feedback. A summary of the appraisal interview (Form 4) was completed by both appraisee and appraiser together. The summary information was collated and fed back anonymously to the appraisers group and PCT highlighting the educational outcomes and identi- fying areas for future improvement. The qualitative and quantitative questionnaire developed specifically for this project explores the attitudes and feelings of appraisees before, during and after the appraisal. It was provided immediately following the appraisal. A copy of the questionnaire may be found in the Appendix. All participants were informed about the aims of the study before receiving their consent. They were also informed of their rights to withdraw from the study without providing a reason (Human Rights Act 1998). All the anonymous questionnaires were then sent to a single named person and were anonymised, and treated as confidential in line with the requirements of the Data Protection Act 1998. Further information on perceived potential learning outcomes was derived from a thematic analysis of the summaries found in Form 4. Con- sent was also sought for the use of these data. The qualitative data were analysed based on the emerging themes from the responses to the ques- tions. The quantitative data were analysed using EXCEL software. The quantitative and qualitative data provided the following breakdown of themes: . feelings prior to the appraisal . quality of feedback received from the appraiser . perceived impact of the appraisal . potential learning outcomes from Form 4s . qualitative analysis on general comments about the appraisal. The paper is written to reflect the above. The researcher and main author (OZ) is not a GP and has no involvement in the GP appraisal pro- cess. The data were analysed by OZ and the emerging themes were corroborated with MR who 446 O Zolle, M Rickenbach and E Gorrod
  • 3. is a GP. EG collected the questionnaires and anonymised them. SS provided some informal feedback in the set-up stage. There may be some bias in terms of using MR’s experience of apprais- als to inform the paper. In this way, any bias incurred by other authors through their experience of appraisals in the PCT was minimised. RESULTS Demographics There were a total number of 107 GPs appraised by the PCT. The breakdown of numbers by type of role and age is provided in Table 1a and b respectively. The division between age groups of GPs has been found to be a useful tool in the development of GP workforce planning considera- tions.19 The response rate was 60% (n=64).20 Appraisees’ feelings prior to the appraisal The results show that the appraisee’s confidence and feeling of comfort with the process increased with exposure to it. Positive comments (Table 2) were received from respondents who had already experienced the appraisal process at least once. Experience of exposure to appraisals might account for the high number of positive responses towards the impending appraisal (66%). One first timer felt that he/she was ‘. . . doubtful of the value of appraisals especially if associated with revalidation’. A third timer reflected that he/she felt ‘. . . not as apprehensive as previously. At last beginning to understand the process’. The com- ments collected from respondents who were familiar with the process show evidence who appraisals were perceived as being ‘. . . motiva- tional . . .’, ‘. . . beneficial . . .’, about ‘. . . achieve- ment . . .’ and about what one is going . . .‘to do next’ even before they take place. Quality of feedback received from the appraiser All 64 respondents (Table 2) felt that the apprais- ers were well prepared, knowledgeable about the process, provided them with a chance to say everything they wanted to say, had a good rapport with the appraisee, and had a good end discussion. The majority of the appraisees (99%; Table 2) felt that the appraiser supported them to think about their developmental needs. This is in spite of the 25% who had previously felt slightly apprehensive about the appraisal. Many of the appraisals took longer than 2 hours (67%). Perceived impact of the appraisal The majority of appraisees felt that the process helped them identify clear learning aims for the next year (84%; Table 3). Only 2% felt that these could have been clearer. Appraisees also felt that the learning aims identified would be easily achieved within the year (58%), and only 7% felt that they would not be able to achieve these with ease. In spite of the challenges imposed by the learning aims, all appraisees felt that these would be completed to some extent within the year. In fact, 82% were somewhat confident in achieving them. Fifty percent of the appraisees felt that the appraisal had contributed to a reasonable feed- back on their work as a GP while some felt that it had contributed either not at all or only slightly (14%). Appraisees felt that the appraisal had helped them to build up confidence as GPs (46%), and 17% felt that the appraisal had not helped in building their confidence. More importantly, some aprraisees felt valued as GPs (58%) as opposed to 19% where the appraisal had no effect on their perceived value as GPs. The appraisees also felt that the appraisal would have an effect on improv- ing patient care (35%). About a third of the respondents felt that the appraisal would either have no effect at all or little effect (29%). Interest- ingly, 8% of the 37 respondents felt that their appraisal would in fact contribute to a delay in retirement age. Potential learning outcomes from Form 4s The categories of learning outcomes identified varied from clinical skills and knowledge and being an effective part of the practice to running a practice and personal, professional life coaching needs (Table 2). The most frequent clinical learn- ing needs identified were: carrying out an audit (28%), recording PUNS and DENS` (22%) and keeping up to date with clinical skills and knowl- edge (14%). Learning outcomes that relate to the running of the practice included: learning about clinical systems (22%), general IT learning (19%) and having practice-based learning in protected Table 1a Breakdown of GP numbers by type GP type Number of GPs GP principals 80 Locum doctors 17 GP retainers 4 Flexible careers scheme 3 Primary Medical Services salaried GPs 3 Table 1b Breakdown of GP numbers by age GP age groups Percentage of GPs Under 34 16 34–49 years 52 50+ years 32 The formative impact of general practice appraisals 447
  • 4. time (11%). The personal or professional life coaching outcomes included retirement planning (8%), finding the right practice (8%) and taking a sabbatical (6%). Qualitative analysis from comments about the appraisal This part of the questionnaire was largely left unanswered by respondents. Those who responded felt positive about the influence that the appraisal could have on them. There were also some suggestions for improvement on the overall appraisal process (Box 1). The perceived educational value of the appraisal is illustrated by the comments collected in Box 1: ‘. . . advised me on how to achieve them (learning needs)’, ‘. . . it helped me focus . . . how my career is progres- sing’. Some suggest a potential impact of apprai- sals on the medical community as a whole: ‘. . . rekindled my enthusiasm to do more teaching work . . .’; ‘. . . useful career advice . . .’. Some respondents are therefore thinking broadly about their work as GPs and about their personal indivi- dual strengths. DISCUSSION We present findings on the impact of appraisals on appraisees using the purely formative model for appraisals. The appraisees in this study show an overall satisfaction with the appraisal model used here and provide evidence for anticipated appraisal impact. Appraisees use words such as ‘inspirational’, ‘motivational’, ‘rekindled my enthu- siasm’, ‘encouraging’ to express the motivational role behind appraisals that is often essential for Table 2 Appraisees’ feelings before the appraisal and the quality of the appraiser’s feedback Before the appraisal Positive (%) Apprehensive (%) Negative (%) Neutral (%) Overall, how did you feel about your 66 25 1 8 impending appraisal? Selection of positive comments ‘. . .not as apprehensive as previously. At last beginning to understand the process.’ ‘I felt much more comfortable. . .’ ‘. . . knew what to expect. . .’ ‘. . . good to spend some time on oneself.’ ‘They always fire me up with some enthusiasm and motivation.’ ‘. . . I enjoy putting together what I have achieved rather what I didn’t get. . .’ . . .’ great. . .to think about what next!!!’ ‘. . . I look forward to it.’ ‘The appraiser is a GP guru.’ ‘Appraisals have always been a beneficial process.’ Selection of less positive comments ‘I felt I had failed to collect enough evidence’ ‘slightly apprehensive. . .’ need to pass’’ ‘I had time off to prepare. . .’ ‘. . . I felt that an in depth reflection of my practise was going to have less relevance. . .’ ‘I am doubtful of the value of appraisals especially if associated with revalidation.’ Quality of the feedback provided by the appraiser Number of respondents out of 64 During the appraisal Yes No Was the appraiser well prepared? 64 0 Was the appraiser knowledgeable about the appraisal process? 64 0 Did you have the chance to say all that you wanted to say? 64 0 Did you feel that you had a good rapport with your appraiser? 64 0 At the end, did the appraiser bring your discussion to a positive conclusion? 64 0 I felt that the appraiser supported me to think about my own development needs and how I can meet them 99% I felt that I was being assessed by the appraiser 1% Would you wish to be appraised by this appraiser again? Yes 99% Unsure 1% No 0% 448 O Zolle, M Rickenbach and E Gorrod
  • 5. any personal developmental experience. The fact that the learning needs that have been identified this year will be reviewed the next also acts as a catalyst to engage in further professional learning activities. The success of the formative appraisal there- fore lies in the discussion between appraiser and appraisee about the individual educational out- comes that may be achieved over the year. It is therefore important in the formative model that appraisees feel they are ‘moving forward’ in their work and careers. Appraisees report in this paper that this form of appraisal could have an impor- tant beneficial impact on patient care. It is the feeling of ‘being valued as a GP’ that is important to many appraisees. We show that the learning outcomes achieved through the formative model of appraisals range from the clinical to the life coaching ones. It is, therefore, not surprising that experienced appraisees look forward to their appraisal as they emphasise each individual’s personal strengths, thereby being a characteristic of the formative model of appraisals. The litera- ture on appraisals is flourishing with reports about adequate appraiser training as appraisals depend on the model of appraisal followed as well as the knowledge and training received by the appraiser. It is interesting to note that appraisers using a combination of summative and formative processes as part of the revalidation model have claimed this to be an inadequate pro- cess that is disengaged from the role of a trainer or educator.21 The model for appraisals that has been sug- gested for England proposes the inclusion and approval of evidence with explicit judgements made about the appraisees’ performance.22 In Scotland, however, a more formative model is fol- lowed based around the five core categories iden- tified for appraisals, allowing the discussion of one category in depth at each appraisal over five years and revalidation is incorporated through participa- tion in an annual developmental appraisal. In the latter model appraisees have been found to derive value and benefit from the formative aspect of the appraisal as they were able to link these to improved practice.23 It has therefore been sug- gested that revalidation and appraisal remain separate.23 In Wales the appraisal process is tightly linked to the delivery of a quality-assured systematic process that manages virtual appraisal folders, selection of appraisers and booking system for appraisal discussions, and manages follow-up actions with the development of personal Table 3 The impact of the appraisal; how the appraisees felt that the appraisal had contributed to their overall educational de- velopment within the areas specified. Perceived appraisal impact by trainees How much do you feel the appraisal has contributed to; Not at 2 (%) 3 (%)4 (%) As much as Number of all possible responses 5 (%) (total of 64 respondents) 1. Clarity of learning aims for the next year. 0 2 14 43 41 44 2. Ease of achieving those aims 7 2 33 46 12 43 3. Likelihood of completing your learning aims for this year 0 3 15 62 20 40 4. Feedback received on your work as a GP. 5 9 36 25 25 44 5. Confidence as a GP 5 12 37 30 16 43 6. How valued do you feel as a GP. 5 14 23 35 23 43 7. Improved patient care 12 17 36 21 14 42 8. A delay in retirement age 68 5 19 3 5 37 Top three more frequent learning outcomes Clinical (on being a doctor) Carrying out an Recording PUNS Keeping up audit & DENS to date Frequency score out of 64 18 14 9 Percentage frequency (%) 28 22 14 Running the practice (on being a businessman) Clinical systems IT learning Practice based learning protected time Frequency score out of 64 14 12 7 Percentage frequency (%) 22 19 11 Personal (on being a human being) Retirement planning Finding the right Taking a practice sabbatical Frequency score out of 64 5 5 4 Percentage frequency (%) 8 8 6 The formative impact of general practice appraisals 449
  • 6. development plans.24 The current process in Wales is formative but there is a debate about how this could be linked to a more summative form of appraisals for revalidation. Overall, the emphasis in the literature so far has been on the provision of adequate training for appraisers, who are not paid any extra income to carry out their roles. It has been reported that the training of peer appraisers is crucial to the effectiveness of the appraisal process.3,7,21 The success of appraisals is partly determined by the quality of the feedback given by a trained professional peer.3,7,21 The role of the appraiser as a source for invaluable specia- lised information is fundamental to the potential impact of the appraisal process. The training that the appraisers received appears to elicit some important positive outcomes from the experience. The majority of appraisees in our study (99%) felt that the appraiser supported them to think about their own developmental needs and how to meet them. Appraisees also value the chance that appraisals offer for discussing issues pertaining to the practice and to their overall careers with a knowledgeable peer. Furthermore, it has been shown that there are inaccuracies found in the assessment of one’s own learning needs and that the identification of gaps in one’s own overall per- formance is not always translated into an effective change in behaviour.23 Well-trained appraisers should contribute to the following developmental challenges experienced by appraisees as identified by Jenkins (2003).22 Those who: . wish to develop but have limited experience of the educational methods for doing so . have limited experience of identifying learning needs Box 1 Positive and Less positive comments from Appraisees regarding the educational and developmental value of the appraisal. Positive educational and developmental comments ‘Made me think about my learning needs and concrete them in written form. . .making them more likely to be achieved.’ ‘I feel more positive about my educational needs.’ ‘. . .identified strengths and weaknesses . . ..’ ‘. . .good in elucidating important areas to include in my portfolio.’ ‘. . .it helped me focus on my clinical and learning needs and how my career is progressing.’ ‘. . .helped prepare my learning needs with ease.’ ‘. . .encouraged me to obtain objectives . . .and think about what I’m learning and achieving.’ ‘. . .guided me through identifying my learning needs/advised me on how to achieve them.’ ‘Useful suggestions for how to achieve my learning outcomes, useful career advice.’ ‘. . .rekindled my enthusiasm to do more teaching work.’ ‘. . .constructive approach to personal development. . .inspiration to get on’ ‘Good opportunity to reflect on my current role. . .’ ‘. . .realistic and productive. . .’ ‘Opportunity to reflect on achievements and positive aspects. . .in the workplace’ ‘. . .cover all areas of development, interest and concern. . .’ ‘Clear thoughtful reflection of my documentation; challenging and supportive.’ ‘Encouraging.’ Total positive comments: 48 out of 64 (75%) Sample of positive educational or developmental comments about the GP in the practice ‘. . .making a realistic plan of how to address the aims and challenges of the practice.’ ‘Good to discuss with a neutral person outside the practice.’ ‘Good. . .to talk to an experienced objective and understanding peer, in confidence.’ ‘Opportunity for confidential discussion about my practices and changes.’ ‘Space to think.’ ‘Chance to speak about my concerns and the stresses I have been under over the past year.’ Less positive issues ‘Using the online toolkit.’ ‘. . .time taken to complete form 4 and then cross examination to the PDP; are both really required?’ ‘I could have been challenged a bit more.’ ‘. . .not really helpful to develop my practice.’ ‘I hate appraisals as my mentality makes me concentrate on the negative. . .’ ‘’. . .some repetition. . .’ ‘. . .get a report from the practitioners in my practice.’ ‘. . .can be repetitive. . .’ ‘. . .process contains elements of jumping through hoops to prove I do the things I already know I do.’ ‘. . .I remain unconvinced about the whole appraisal process. It is bureaucratic and I do not think generally helpful. A lot of form filling. . .little added self reflection.’ ‘I think mentoring would be more appropriate.’ Total less positive comments: 15 out of 64 (23%) 450 O Zolle, M Rickenbach and E Gorrod
  • 7. . have difficulty in separating learning aims from objectives . find it difficult to recognise the usefulness of com- plaints for exploring learning needs . understand how to identify learning needs but dis- like recording the information . find it difficult to achieve or prioritise all their learning objectives . find it difficult to commit themselves to areas of learning and self-development. The uniqueness of appraisals lies precisely in the ability to provide professional guidance in all of the above areas. No other current form of profes- sional development or formative assessment for GPs provides for this directly tailored to the indi- vidual. Learning objectives should be achievable and realistic but they should also pose an educa- tional and/or a developmental challenge. Our findings on the feeling of confusion related to the nature of appraisals and their overall edu- cational value are in agreement with the litera- ture.3,5,16 Limitations to the study . We have no control groups for this study and no assessment of the non-responder group. . We may only report the potential impact of appraisals as perceived by the appraisees and as revealed from their learning outcomes. . We cannot assume that the majority of the apprais- ees will have achieved their learning goals. . It is also possible that the appraisee might have been favourably biased in their response as affected by the relationship and time spent with the appraiser. Implications for future research and clinical practice The GMC has proposed a system that will encompass both appraisal and assessment for revalidation but if there were to be a combined system there would be a need to ensure that the professional development focus and the educa- tional impact brought in by the current appraisal system are not lost. McKinstry et al (2005)3 have shown that appraisees are concerned with the current links that are being made with revalida- tion. These concerns stem from the potential for clashes between the appraiser and the appraisee if the system for revalidation were to be imposed as part of the appraisal process. Appraisals are not about poor performance, they are about improving the performance of all and for the ben- efit of both the public and the medical commu- nity. Funding This study received no funding. The researcher and main writer for this study, OZ, is employed by NHS Education South Central. The Wessex Dean- ery is part of NHS Education South Central. Ethical approval None required. All the data received from partici- pants were anonymised from the source. Conflicts of interest None. References 1 West MA, Borrill C and Dawson J (2006) The link between the management of employees and patient mortality in acute hospitals. International Journal of Human Resources Management 13: 1299–310. 2 Adams R, Iling J, Jelley D, Walker C and van Zwanen- berg T (2005) The critical success factors in delivering GP appraisal: a review of the first year in Northumber- land and Tyne and Wear; Sunderland Teaching PCT. Postgraduate Institute for Medicine and Dentistry: Sun- derland. 3 McKinstry B, Peacock H and Shaw J (2005) Experiences of partner and external peer appraisal. British Journal of General Practice 55: 539–43. 4 Boylan O, Bradley T and McKnight A (2005) GP percep- tions of appraisal: professional development, perfor- mance management, or both? British Journal of General Practice 55(516): 544–5. 5 Middlemas J and Siriwardena AN (2003) General practi- tioners, revalidation and appraisal: a cross sectional survey of attitudes, beliefs and concerns in Lincolnshire. Medical Education 37: 778–85. 6 Chambers R, Tavabie A and See S (2003) Exploring ideas for the evaluation of the GP appraisal system. Education for Primary Care 14: 579–83. 7 Lewis M, Elwyn G and Wood F (2003) Appraisal of family doctors: an evaluation study. British Journal of General Practice 53: 454–60. 8 Dhaliwal JS (2003) Why appraisal is not performance management – or is it? Education for Primary Care 14: 555–7. 9 Conlon M (2003) Appraisal: the catalyst of personal development. BMJ 327: 389–91. 10 General Medical Council (2001) Good Medical Practice. GMC: London. 11 Martin D, Harrison P, Joesbury H and Wilson R (2006) Appraisal for GPs. School of Health and Related Research, University of Sheffield. 12 Department of Health. CMO Update, Issue 44. www.dh.gov.uk/cmo 13 Sharpe B (2003) Revalidation and the quality agenda. Education for Primary Care 14: 543–7. 14 Thomas P (2005) Will GP revalidation make things better or worse? British Journal of General Practice 318–19. 15 Mohanna K (2003) Appraisal: what now? Education for Primary Care 14: 537–9. 16 Torjesen I (2006) Moral dilemma for rural GPs. Pulse, 16 November. 17 Bruce D, Phillips K, Reid R, Snadden D and Harden R The formative impact of general practice appraisals 451
  • 8. (2004) Revalidation for general practitioners: rando- mised comparison of two revalidation models. BMJ 328: 687–91. 18 Gorrod E (2006) Annual Report on GP Appraisal in East- leigh and TVS PCT 2005–2006. Eastleigh and Test Valley South PCT: Hampshire. 19 Smith F, Scallan S, Bell J and Curson J (2006) Training for General Practice in Hampshire & the Isle of Wight: 2006–2010. A Severn and Wessex Deanery Report. 20 Oppenheim AN (1992) Questionnaire Design, Interview- ing and Attitude Measurement. Continuum International Publishing: London. 21 Baron R, Agius S and Lewis B (2007) GP appraisal: per- ceptions of appraisers and appraisees. Education for General Practice 18: 373–81. 22 Jenkins M (2003) Guide to utter failure: a tool for provid- ing insight into the appraisal process. Education for Pri- mary Care 14: 569–71. 23 Wakeling J, Cameron N and Lough M (2008) What factors impact on general practitioners’ engagement with appraisal? A qualitative evaluation in Scotland. Educa- tion for Primary Care 19: 615–23. 24 Rhydderch M, Langharne K, Manfell N, Powncwby P and Lewis M (2008) Developing a skills based model to pro- mote effective appraisal discussions amongst GPs in Wales. Education for Primary Care 19: 496–505. Correspondence to: Dr Olga Zolle, Romsey Road, High- croft, Winchester SO22 5DH, UK. Email: Olga.Zolle@ nesc.nhs.uk Accepted October 2009 452 O Zolle, M Rickenbach and E Gorrod
  • 9. Appendix APPRAISEE FEEDBACK FORM Appraisee: Appraiser: Before the appraisal . . . Overall, how did you feel about your impending appraisal? Positive Apprehensive Negative Neutral Please add comments if desired: During the appraisal . . . Was the appraiser well prepared? Yes No Was the appraiser knowledgeable about the appraisal process? Yes No Did you have the chance to say all that you wanted to say? Yes No Did you feel that you had a good rapport with your appraiser? Yes No At the end, did the appraiser bring your discussion to a positive conclusion? Yes No How long did your appraisal meeting last? Less than 1 hour 1–2 hours More than 2 hours Overall, which of these two statements more closely reflects your experience of this interview? I felt that the appraiser supported me to think I felt that I was being assessed by the about my own development needs and how appraiser I can meet them Would you wish to be appraised by this appraiser again? Yes No If you have answered ‘No’ to any of the questions in this section, please explain why you feel this: The formative impact of general practice appraisals 453
  • 10. Since you have been undergoing appraisal . . . On a scale of 1 to 5, where 1 = not at all and 5 = as much as possible, please rate how much do you feel appraisal has contributed to: a) How valued you feel as a general practitioner b) The feedback you receive on your work as a general practitioner c) Your confidence as a general practitioner d) The clarity of your learning aims for the next year e) The ease of achieving those aims f) The likelihood of you completing your learning aims for this year g) A delay in your retirement age h) Improved patient care General . . . What was good about your appraisal? What aspects did you find unproductive; could your appraiser have done anything differently to help? 454 O Zolle, M Rickenbach and E Gorrod