introduction of 360 degree appraisals 1
Here is a simple guide for introducing 360 degree appraisals into an organization (and any other
management system for that matter):
• Choose/design a system (or system provider), ie., research and investigate your options
(other local or same-sector companies using 360 already are a helpful reference point, or
your trade association HR group, or a specialist HR advisory body such as CIPD in the UK if
you are a member).
• When you've decided on a system, pilot it with a few people to make sure it does what you
expect. (It's best to establish some simple parameters or KPI's by which you can make this
assessment, rather than basing success on instinct or subjective views.)
• When satisfied with the system, launch it via a seminar or workshop, preferably including
role-plays and/or practical demonstration.
• Support the implementation with ongoing training, (include an overview in your induction
training as well), a written process guide/booklet, and also publish process and standards
on your intranet if you have one.
• Establish review and monitoring responsibility.
• Ensure any 360 degree appraisal system system is applied from top down, not bottom up,
so everyone can see that the CEO is happy to undertake what he/she expects all the other
staff to do. (As with anything else, if the CEO and board agrees to undertake it first, the
system will have much stronger take-up and credibility.)
performance appraisals
performance appraisals, performance evaluation and
assessment of job skills, personality and behaviour - and
tips for '360 degree feedback' and '360° appraisals'
Performance appraisals are essential for the effective management and evaluation of staff.
Appraisals help develop individuals, improve organizational performance, and feed into business
planning. Formal performance appraisals are generally conducted annually for all staff in the
organization. Each staff member is appraised by their line manager. (Directors are appraised by
the CEO, who is appraised by the chairman or company owners, depending on the size and
structure of the organization). Annual performance appraisals enable management and
monitoring of standards, agreeing expectations and objectives, and delegation of responsibilities
and tasks. Staff performance appraisals also establish individual training needs and enable
organizational training needs analysis and planning. Performance appraisals data feeds into
organizational annual pay and grading reviews, and coincides with the business planning for the
next trading year. Performance appraisals generally review each individual's performance against
objectives and standards for the trading year, agreed at the previous appraisal meeting.
Performance appraisals are also essential for career and succession planning. Performance

appraisals are important for staff motivation, attitude and behaviour development, communicating
organizational
2
aims, and fostering positive relationships between management and staff. Performance appraisals
provide a formal, recorded, regular review of an individual's performance, and a plan for future
development. In short, performance and job appraisals are vital for managing the performance of
people and organizations.
are performance appraisals truly beneficial?
It is sometimes fashionable in the 'modern age' to dismiss traditional processes such as
performance appraisals as being irrelevant or unhelpful. Be very wary however if considering to
remove appraisals from your own organisational practices. It is likely that the critics of the
appraisal process are the people who can't conduct them very well. It's a common human
response to want to jettison something that one finds difficult. Appraisals - in whatever form, and
there are various - have been a mainstay of management for decades, for good reasons.
Think about everything that performance appraisals can achieve and contribute to when they are
properly managed, for example:
• performance measurement - transparent, short, medium and long term
• clarifying, defining, redefining priorities and objectives
• motivation through agreeing helpful aims and targets
• motivation though achievement and feedback
• training needs and learning desires - assessment and agreement
• identification of personal strengths and direction - including unused hidden strengths
• career and succession planning - personal and organisational
• team roles clarification and team building
• organisational training needs assessment and analysis
• appraisee and manager mutual awareness, understanding and relationship
• resolving confusions and misunderstandings
• reinforcing and cascading organisational philosophies, values, aims, strategies, priorities,
etc
• delegation, additional responsibilities, employee growth and development
• counselling and feedback
• manager development - all good managers should be able to conduct appraisals well - it's
a fundamental process
• the list goes on..
People have less and less face-to-face time together these days. Performance appraisals offer a
way to protect and manage these valuable face-to-face opportunities. My advice is to hold on to

and nurture these situations, and if you are under pressure to replace performance appraisals
with some sort of (apparently) more efficient and cost effective methods, be very sure that you
can safely cover all the aspects of performance and attitudinal development that a well-run
performance appraisals system is naturally designed to achieve.
There are various ways of conducting performance appraisals, and ideas change over time as to
what are the most effective appraisals methods and systems. Some people advocate traditional
appraisals and forms; others prefer 360-degree-type appraisals; others suggest using little more
than a blank sheet of paper.
In fact performance appraisals of all types are effective if they are conducted properly,
and better still if the appraisal process is clearly explained to, agreed by, the people
involved.
Managers need guidance, training and encouragement in how to conduct appraisals
properly. Especially the detractors and the critics. Help anxious managers (and directors)
develop and adapt appraisals methods that work for them. Be flexible. There are lots of ways to
conduct appraisals, and particularly lots of ways to diffuse apprehension and fear - for managers
and appraisees alike. Particularly - encourage people to sit down together and review informally
and often - this removes much of the pressure for managers and appraisees at formal appraisals
times. Leaving everything to a single make-or-break discussion once a year is asking for trouble
and trepidation.
Look out especially for the warning signs of 'negative cascaded attitudes' towards appraisals.
This is most often found where a senior manager or director hates conducting appraisals, usually
because they are uncomfortable and inexperienced in conducting them. The senior
manager/director typically will be heard to say that appraisals don't work and are a waste of time,
which for them becomes a self-fulfilling prophecy. This attitude and behaviour then cascades
down to their appraisees (all the people in their team) who then not surprisingly also apply the
same 'no good - not doing it' negative attitude to their own appraisals responsibilities (teams).
And so it goes. A 'no good - not doing it' attitude in the middle ranks is almost invariably traceable
back to a senior manager or director who holds the same view. As with anything, where people
need help doing the right thing, help them.
All that said, performance appraisals that are administered without training (for those who
need it), without explanation or consultation, and conducted poorly will be counter-
productive and are a waste of everyone's time.
Well-prepared and well-conducted performance appraisals provide unique
opportunities to help appraisees and managers improve and develop, and thereby also
the organisations for whom they work.
Just like any other process, if performance appraisals aren't working, don't blame the
process, ask yourself whether it is being properly trained, explained, agreed and
conducted.

effective performance appraisals
Aside from formal traditional (annual, six-monthly, quarterly, or monthly) performance appraisals,
there are many different methods of performance evaluation. The use of any of these methods
depends on the purpose of the evaluation, the individual, the assessor, and the environment.
The formal annual performance appraisal is generally the over-riding instrument which gathers
together and reviews all other performance data for the previous year.
Performance appraisals should be positive experiences. The appraisals process provides the
platform for development and motivation, so organizations should foster a feeling that
performance appraisals are positive opportunities, in order to get the best out of the people and
the process. In certain organizations, performance appraisals are widely regarded as something
rather less welcoming ('bollocking sessions' is not an unusual description), which provides a basis
only on which to develop fear and resentment, so never, never, never use a staff performance
appraisal to handle matters of discipline or admonishment, which should instead be handled via
separately arranged meetings.
types of performance and aptitude assessments,
including formal performance appraisals
• Formal annual performance appraisals
• Probationary reviews
• Informal one-to-one review discussions
• Counselling meetings
• Observation on the job
• Skill- or job-related tests
• Assignment or task followed by review, including secondments (temporary job cover or
transfer)
• Assessment centres, including observed group exercises, tests presentations, etc.
• Survey of opinion of others who have dealings with the individual
• Psychometric tests and other behavioural assessments
• Graphology (handwriting analysis)
None of these methods is mutually exclusive. All of these performance assessment methods can
be used in conjunction with others in the list, depending on situation and organizational policy.
Where any of these processes is used, the manager must keep a written record, and must ensure
agreed actions are followed up. The notes of all review situations can then be referred to at the
formal appraisal.

Holding regular informal one-to-one review meetings greatly reduces the pressure and time
required for the annual formal appraisal meeting. Holding informal reviews every month is ideal all
staff. There are several benefits of reviewing frequently and informally:
• The manager is better informed and more up-to-date with his or her people's activities
(and more in touch with what lies beyond, e.g., customers, suppliers, competitors,
markets, etc)
• Difficult issues can be identified, discussed and resolved quickly, before they become more
serious.
• Help can be given more readily - people rarely ask unless they see a good opportunity to
do so - the regular informal review provides just this.
• Assignments, tasks and objectives can be agreed completed and reviewed quickly - leaving
actions more than a few weeks reduces completion rates significantly for all but the most
senior and experienced people.
• Objectives, direction, and purpose is more up-to-date - modern organizations demand
more flexibility than a single annual review allows - priorities often change through the
year, so people need to be re-directed and re-focused.
• Training and development actions can be broken down into smaller more digestible chunks,
increasing success rates and motivational effect as a result.
• The 'fear factor', often associated by many with formal appraisals, is greatly reduced
because people become more comfortable with the review process.
• Relationships and mutual understanding develops more quickly with greater frequency of
meetings between manager and staff member.
• Staff members can be better prepared for the formal appraisal, giving better results, and
saving management time.
• Much of the review has already been covered throughout the year by the time comes for
the formal appraisal.
• Frequent review meetings increase the reliability of notes and performance data, and
reduces the chances of overlooking things at the formal appraisal.
performance appraisals process
• Prepare - prepare all materials, notes agreed tasks and records of performance,
achievements, incidents, reports etc - anything pertaining to performance and achievement
- obviously include the previous performance appraisal documents and a current job
description. A good appraisal form will provide a good natural order for proceedings, so use
one. If your organization doesn't have a standard appraisal form then locate one, or use
the template below to create one, or download the new performance appraisal form from
the free resources section, (which also contains other useful tools for appraisals and
developing people - you'll need Acrobat Reader to view and download, available free from
Adobe.com). The appraisal form is also available free in MSWord. Whatever you use,
ensure you have the necessary approval from your organization, and understand how it
works. Organize your paperwork to reflect the order of the appraisal and write down the
sequence of items to be covered. If the appraisal form includes a self assessment section
and/or feedback section (good ones do) ensure this is passed to the appraisee suitably in

advance of the appraisal with relevant guidance for completion. A sample performance
appraisal template is available free below, which you can use as is or adapt to create your
own form.
• Inform - inform the appraisee - ensure the appraisee is informed of a suitable time and
place (change it if necessary), and clarify purpose and type of appraisal - give the
appraisee the chance to assemble data and relevant performance and achievement records
and materials. If the appraisal form does not imply a natural order for the discussion then
provide an agenda of items to be covered.
• Venue - ensure a suitable venue is planned and available - private and free from
interruptions - observe the same rules as with recruitment interviewing - avoid hotel
lobbies, public lounges, canteens - privacy is absolutely essential (it follows also that
planes, trains and automobiles are entirely unsuitable venues for performance
appraisals......)
• Layout - room layout and and seating are important elements to prepare also - don't
simply accept whatever layout happens to exist in a borrowed or hired room - layout has a
huge influence on atmosphere and mood - irrespective of content, the atmosphere and
mood must be relaxed and informal - remove barriers - don't sit in the boss's chair with the
other person positioned humbly on the other side of the desk; you must create a relaxed
situation, preferably at a meeting table or in easy chairs - sit at an angle to each other, 90
degrees ideally - avoid face to face, it's confrontational.
• Introduction - relax the appraisee - open with a positive statement, smile, be warm and
friendly - the appraisee may well be terrified; it's your responsibility to create a calm and
non-threatening atmosphere. Set the scene - simply explain what will happen - encourage
a discussion and as much input as possible from the appraisee - tell them it's their
meeting not yours. Confirm the timings, especially finishing time. If helpful and appropriate
begin with some general discussion about how things have been going, but avoid getting
into specifics, which are covered next (and you can say so). Ask if there are any additional
points to cover and note them down so as to include them when appropriate.
• Review and measure - review the activities, tasks, objectives and achievements one by
one, keeping to distinct separate items one by one - avoid going off on tangents or vague
unspecific views. If you've done your preparation correctly you will have an order to follow.
If something off-subject comes up then note it down and say you'll return to it later (and
ensure you do). Concentrate on hard facts and figures, solid evidence - avoid conjecture,
anecdotal or non-specific opinions, especially about the appraisee. Being objective is one of
the greatest challenges for the appraiser - as with interviewing, resist judging the
appraisee in your own image, according to your own style and approach - facts and figures
are the acid test and provide a good neutral basis for the discussion, free of bias and
personal views. For each item agree a measure of competence or achievement as relevant,
and according to whatever measure or scoring system is built into the appraisal system.
This might be simply a yes or no, or it might be a percentage or a mark out of ten, or an A,
B, C. Reliable review and measurement requires reliable data - if you don't have the
reliable data you can't review and you might as well re-arrange the appraisal meeting. If a
point of dispute arises, you must get the facts straightened out before making an
important decision or judgement, and if necessary defer to a later date.

• Agree an action plan - An overall plan should be agreed with the appraisee, which
should take account of the job responsibilities, the appraisee's career aspirations, the
departmental and whole organization's priorities, and the reviewed strengths and
weaknesses. The plan can be staged if necessary with short, medium and long term
aspects, but importantly it must be agreed and realistic.
• Agree specific objectives - These are the specific actions and targets that together form
the action plan. As with any delegated task or agreed objective these must adhere to the
SMARTER rules - specific, measurable, agreed, realistic, time-bound, enjoyable, recorded.
If not, don't bother. The objectives can be anything that will benefit the individual, and that
the person is happy to commit to. When helping people to develop, you are not restricted
to job-related objectives, although typically most objectives will be.
• Agree necessary support - This is the support required for the appraisee to achieve the
objectives, and can include training of various sorts (external courses and seminars,
internal courses, coaching, mentoring, secondment, shadowing, distance-learning, reading,
watching videos, attending meetings and workshops, workbooks, manuals and guides;
anything relevant and helpful that will help the person develop towards the standard and
agreed task. Be careful to avoid committing to training expenditure before suitable
approval, permission or availability has been confirmed - if necessary discuss likely training
requirements with the relevant authority before the appraisal to check. Raising false hopes
is not helpful to the process.
• Invite any other points or questions - make sure you capture any other concerns.
• Close positively - Thank the appraisee for their contribution to the meeting and their
effort through the year, and commit to helping in any way you can.
• Record main points, agreed actions and follow-up - Swiftly follow-up the meeting
with all necessary copies and confirmations, and ensure documents are filed and copied to
relevant departments, (HR, and your own line manager typically).
performance appraisal form template
This performance appraisal template has been specially developed to adapt for your own
situation. For many situations the example will serve perfectly well without amending. Obviously
you'll need to create a bit of space for the answers to the questions. Here is a free performance
appraisal form in pdf format, and here is the same performnce appraisal form in MSWord format.
These free forms are based on the template below.
performance appraisal form template/sample
organization, division and department:
year or period covered:

name:
age:
position:
location/site/based at:
months in present position:
length of service:
Part A (to be completed by the appraisee before the interview and sent to the appraiser x days
before the appraisal)
A1 State your understanding of your duties and responsibilities.
A2 Discussion points: (not exhaustive or definitive - for more ideas look at the interviews
questions)
1. Has the past year been good/bad/satisfactory or otherwise for you, and why?
2. What do you consider to be your most important achievements of the past year?
3. What do you like and dislike about working for this organization?
4. What elements of your job do you find most difficult?
5. What elements of your job interest you the most, and least?
6. What do you consider to be your most important tasks in the next year?
7. What action could be taken to improve your performance in your current position by you,
and your boss?
8. What kind of work or job would you like to be doing in one/two/five years time?
9. What sort of training/experience would benefit you in the next year?
A3 List the objectives you set out to achieve in the past 12 months (or the period covered by this
appraisal) with the measures or standards agreed - against each comment on achievement or
otherwise, with reasons where appropriate. Score the performance against each objective (1-3 =
poor, 4-6 = satisfactory, 7-9 = good, 10 = excellent):
A4 Score your own capability or knowledge in the following areas in terms of your current role
requirements (1-3 = poor, 4-6 = satisfactory, 7-9 = good, 10 = excellent). If appropriate bring
evidence with you to the appraisal to support your assessment. (NB This list is not exhaustive or
definitive - the list should reflect the requirements of the job and the career path. See also the
skills and behaviour assessment tool for other aspects to include in this list.)
1. commercial judgement
2. product/technical knowledge
3. time management
4. planning, budgeting and forecasting

5. reporting and administration
6. communication skills
7. delegation skills
8. IT/equipment/machinery skills
9. meeting expectations, deadlines and commitments
10. creativity
11. problem-solving and decision-making
12. team-working and developing/helping others
13. energy, determination and work-rate
14. steadiness under pressure
15. leadership and integrity
16. adaptability, flexibility, and mobility
17. personal appearance and image
A5 In light of your current capabilities, your performance against past objectives, and your future
personal growth and/or job aspirations, what activities and tasks would you like to focus on
during the next year.
Part B (to be completed during the appraisal by the appraiser - where appropriate and safe to do
so, certain items can completed by the appraiser before the appraisal, and then discussed and
validated or amended in discussion with the appraisee during the appraisal.)
name of appraiser:
position:
time managing appraisee:
B1 Describe the purpose of the appraisee's job. Discuss and compare with self-appraisal
entry in A1. Clarify job purpose and priorities where necessary.
B2 Review discussion points in A2, and note the points of interest and action.
B3 List the objectives that the appraisee set out to achieve in the past 12 months (or the period
covered by this appraisal - typically these objectives will have been carried forward from the
previous appraisal record) with the measures or standards agreed - against each comment on
achievement or otherwise, with reasons where appropriate. Score the performance against each
objective (1-3 = poor, 4-6 = satisfactory, 7-9 = good, 10 = excellent). Compare with the self-
appraisal in B3. Discuss and note points of interest and action, particularly training
and development needs and wishes.
B4 Score the appraisee's capability or knowledge in the following areas in terms of their current
(and if known, next) role requirements (1-3 = poor, 4-6 = satisfactory, 7-9 = good, 10 =
excellent). (NB This list is not exhaustive or definitive - the list should reflect the requirements of
the job and the career path. See also the skills and behaviour assessment tool for other aspects to

include in this list.) Compare with the self-appraisal in B4. Discuss and note points of
interest and action, particularly training and development needs and wishes.
1. commercial judgement
2. product/technical knowledge
3. time management
4. planning, budgeting and forecasting
5. reporting and administration
6. communication skills
7. delegation skills
8. IT/equipment/machinery skills
9. meeting expectations, deadlines and commitments
10. creativity
11. problem-solving and decision-making
12. team-working and developing/helping others
13. energy, determination and work-rate
14. steadiness under pressure
15. leadership and integrity
16. adaptability, flexibility, and mobility
17. personal appearance and image
B5 Discuss and agree the appraisee's career direction options and wishes, and readiness for
promotion, and compare with and discuss the self-appraisal entry in A5. (Some people do
not wish for promotion, but everyone is capable of, and generally benefits from, personal
development - development and growth should be available to all, not just the ambitious).
B6 Discuss and agree the skills, capabilities and experience required for competence in current
role, and if appropriate for readiness to progress to the next role or roles. (It is usually helpful to
refer to the skill-set or similar to that shown in A/B4, in order to accurately identify all
development areas, whether for competence at current level or readiness to progress to next job
level/type.)
B7 Discuss and agree the specific objectives that will enable the appraisee to reach competence
and to meet required performance in current job. These must adhere to the SMARTER rules -
specific, measurable, agreed, realistic, time-bound, enjoyable, recorded.
B8 Discuss and agree the specific objectives that will enable the appraisee to move towards, or
achieve readiness for, the next job level/type, or if no particular next role is identified or sought,
to achieve the desired personal growth or experience. Must also adhere to SMARTER rules.
B9 Discuss and agree as far as is possible (given budgetary, availability and authorisation
considerations) the training and development support to be given to help the appraisee meet the
agree objectives.
NB Appraisers should note that personal development and support must be offered to all
employees, not just the ambitious. Training isn't restricted to sending someone on an external

course - it includes internal courses, coaching, mentoring (mentoring someone else and well as
being mentored), secondment to another role (eg deputising for someone while they are away on
holiday), shadowing, distance-learning, reading books, watching videos, attending meetings and
workshops, workbooks, manuals and guides, researching, giving presentations; anything relevant
and helpful that will help the person develop towards the standard and agreed task. Be careful to
avoid committing to training expenditure before suitable approval, permission or availability has
been confirmed - if necessary discuss likely training requirements with the relevant authority
before the appraisal to check. Raising false hopes is not helpful to the process.
B10 Any other issues (Ideally and hopefully none, but it's important to offer the opportunity to
the appraisee to raise any other points, even if they need to be discussed at another meeting,
outside of the appraisal process, which would generally be the case.)
Signed and dated appraiser and appraisee:
(Finally it's advisable to show instructions as to the distribution of copies of the completed form, a
reminder of its confidential nature, and a statement as to the individual's rights under the data
protection laws applicable.)
appraisals references and tools
Some of these are pdf files, available to view and download free - for which you'll need Adobe
Acrobat Reader - available free from adobe.com.
performance appraisal form sample/template - downloadable pdf
skill set and behaviour set assessment tool - see instructions for use below
training needs analysis tool - see instructions for use below - also management training needs
analysis spreadsheet tool in Excel
coaching and training process flow diagram
delegation/objectives SMART tool
training planner tool
Kolb learning styles theory and diagram
empathy to build trust and diffuse conflict
transactional analysis - for better communicating and understanding
delegation - how to

Adams equity theory on job motivation and diagram
job interviewing (the process is similar to appraisals and many of the questions are useful and can
be adapted for the appraisal meeting or the appraisal form itself)
leadership tips
job descriptions (helpful at appraisals)
how to use skill-set assessments and training needs
analysis tools
The skill/behaviour set and training needs analysis tools (available in pdf and working file MSExcel
formats from the free resources section) are simple, effective and flexible assessments and
analysis tools for performing individual and group Training Needs Analysis.
The Training Needs Analysis (TNA) spreadsheet in its two different variations (sales/commercial,
and management) is a free and yet immensely powerful tool for identifying, assessing, analysing,
prioritising and planning trining needs, for small teams, and very large organisations
Here is a free working example of a Skill Set Assessment for a commercial/sales role, in MSExcel.
Here is a free working example of the Training Needs Analysis Tool for the sales/commercial role ,
in MSExcel.
Also: Manager Skill Set Assessment Tool in Excel,
and Training Needs Analysis Tool for Management Role in Excel.
You can use the tools as they are or adapt them to suit your situation. Obviously ensure that the
skill descriptions are consistent throughout the individual assessment tool and the Training Needs
Analysis tool, although it is entirely possible to include a variety of skill-sets on a single TNA
spreadsheet.
You can use whatever scoring system suits you and your situation, although number scoring
(rather than words or letters) is necessary for spreadsheet analysis.
A 1-4 scoring system generally works well, since it gives less opportunity for middling, non-
committal answers. Primarily we need to know simply whether capability is adequate for the
role or not.

Ensure you identify clear definitions for the scoring, particularly if comparing or analysing different
people's scores, where consistency of measurement is important, eg:
• 1 = little or no competence
• 2 = some competence, but below level required for role
• 3 = competence at required level for role
• 4 = competence exceeds level required for role
Or:
O 1 = never meets standard
2 = sometimes meets standard
3 = often meets standard
4 = always meets standard
For self-use: The skills/behaviour set assessments require some interpretation and ideally
discussion with a trusted friend, colleague or boss to establish the 2nd view validation. As well as
encouraging self-awareness development and simply thinking about one's own feelings and
aptitudes, the assessment and reflection are an interesting and viable basis for
assessing/discussing/reviewing personal development and career focus. When the scoring is
completed you can prioritise your development needs (essential skills with the lowest scores).
For use with others as development tool: The skill/behaviour set is an effective tool for
recruitment, appraisals and ongoing development and training. It can be adapted for different
roles, and if used with existing staff ideally the person performing the role must have input to the
skill and behavioural criteria listed, and the importance (essential or desirable) for each
characteristic in the role. Working with a group to adapt the skillset criteria according to the
people's jobs makes an interesting workshop and team building session: involving people in
developing the system creates a sense of ownership and commitment to using the assessment
method itself. The skillset/behavioural tests can be used in conjunction with the Training Needs
Analysis tool available from the website as a pdf or as a working Excel file from the free resources
section. Assessment can be carried out formally one-to-one as part of an appraisal or review
meeting, referring to evidence if appropriate, or informally in a workshop situation as a group
exercise (assessment in pairs, with partners helping to establish the 2nd view validation for each
other). Whether informally or formally assessed, the results for a group can be transferred to the
Training Needs Analysis tool, to identify group training priorities. Training priorities are the
essential skills with the lowest average scores.
Informal assessments in a workshop situation also enable an immediate 'straw poll' analysis of
group training needs, and as such provide an excellent method for quickly identifying and
agreeing training and development needs for a group.

tips on scoring systems for skills audits, appraisals and
training needs analysis
Scoring and measuring system suitability is critical, especially if you are making big decisions on
the outcomes, which require clear score definitions and implications (explain to participants the
judgements/actions which will stem from the scoring).
Generally a score range of 1-3 is too narrow. Not only because life isn't that simple, but mainly
because the mid-way 2 option encourages fence-sitting which inhibits clarity of individual and
overall results (as any odd number score range tends to do). 1-3 or 1-5 virtually ensures you end
up with a cloudy result because so many answers are in the middle.
If you need to change from a 3 or 5 point system, this objective-scientific angle might provide you
with the best lever to do so. 1-4 is much better because people have to decide whether the ability
is to standard or not - there's not an automatic average or mid-way for the 'don't knows'.
If you have to stick with 1-3 then ensure the meanings are such as to ensure black or white
answers.
'Grey' answers at number 2 in a 1-3 scale, eg., average, medium, satisfactory, etc., aren't really
any help. Nor are the typical definitions found at number three in a 1-5 scale.
A way of making a 1-3 scale acceptable is:
• 1 - needs improving
• 2 - good
• 3 - excellent
Here the 1-3 is effectively turned into a 1-2 (yes/no or is/isn't) scoring system (whereby 1 & 2 =
above standard; 3 = below standard) which at least enables a clear decision, albeit just yes or no,
which in actual fact is all that's necessary for many TNA's.
Tight scales are fine - in fact in some ways easier - for a group training needs analysis, but are
not good for individual skills audits or training needs analysis, where the question of degree is
more important for individual task direction and development planning, and to enable more
reliable comparison between individuals.
The accuracy and reliability of any scoring system increases with full description/definitions, and
better still with examples for each score band. This gives everyone the same objective-scientific
reference points, and reduces subjectivity.
tips on 360 degree feedback and 360 appraisals

360 degree appraisals are a powerful developmental method and quite different to traditional
manager-subordinate appraisals (which fulfil different purposes). As such a 360 degree process
does not replace the traditional one-to-one process - it augments it, and can be used as a stand-
alone development method.
360 degree appraisals involve the appraisee receiving feedback from people (named or
anonymous) whose views are considered helpful and relevant. The feedback is typically provided
on a form showing job skills/abilities/attitudinal/behavioural criteria and some sort of scoring or
value judgement system. The appraisee should also assess themselves using the same feedback
instrument or form.
360 degree respondents can be the appraisee's peers, up-line managers/execs, subordinate staff,
team members, other staff, customers, suppliers - anyone who comes into contact with the
appraisee and has opinions/views/reactions of and to the appraisee. Numerous systems and
providers are available - I wouldn't recommend any in particular because my view about this
process is that you should develop a process and materials for your own situation, preferably
involving the appraisees in this, which like all participative approaches, often works well.
You can develop your own 360 degree feedback system by running a half-day or full day
workshop (depending on extent and complexity of the required process) involving the appraisees
or a sample group, during which process and materials can be created and provisionally drafted.
The participative workshop approach as ever will give you something that's wholly appropriate
and 'owned' instead of something off-the-shelf or adapted, which would be arbitrary, mostly
inappropriate and impracticable (in terms of criteria and process), and 'not invented here', ie.,
imposed rather than owned.
I would recommend against restricting the 360 feedback to peers and managers only - it's a
waste of the potential of the 360 degree appraisal method. To use the feedback process for its
fullest '360 degree' benefit involve customers (in the broadest sense - could be patients, students,
users, depending on the organization), staff, suppliers, inspectors, contractors, and others for
whom good working relationships and understanding with the appraisee affect overall job
performance, quality, service, etc.
Developing 360 degree appraisals systems process make ideal subjects for a workshops, which in
itself contains some very helpful developmental benefits and experience for all involved. If you're
not able to get everyone together for a workshop you should solicit input and ideas - particularly
about appraisal criteria and respondents and anonymity - then draft out process and materials -
then issue for approval, then pilot, review, adapt and then implement. Adapt, improve and
develop on an ongoing basis.
It is my view that no aspects of 360 feedback should ever be mandatory for any appraisee or
respondent. Given more than three or four similar role-types being appraised it's not sensible to
produce individually tailored criteria, in which case when it comes to the respondents completing
the feedback not all the criteria will be applicable for all respondents, nor for all appraisees either.
By the same when designing the feedback instruments (whether hard-copy documents or online
materials), it's useful to allow space for several 'other' aspects that the appraisee might wish to

add to the standard criteria, and space for respondents to add 'other' comments. Open honest
feedback can touch sensitivities, so be sure that appraisees understand and agree to the criteria,
respondents (by type, if not named) and process.
Ensure suitable and sensitive counselling is provided as part of the informing of feedback results.
If 360 degree feedback results are to be analysed collectively to indicate the overall/total situation
(ie., to assist in determining organizational training and development needs for instance), think
carefully about the feedback form scoring system and particularly its suitability for input to some
sort of analysis tool, which could be a spreadsheet, and therefore numerically based requiring
numerical scores, rather than words, (words of course are more difficult to count and measure,
and while words and description assessment enables more subtlety, they also allow more room
for misunderstanding and misinterpretation).
For guidance have a look at the skills and behavioural assessment tool - it's not a 360 degree tool,
but is an example of the basis of one, and some of the skills elements that can be included in a
360 degree appraisals form.
Similarly the training needs analysis tool is an example of a collective or organizational
measurement tool, based on the input of a number of individual feedback assessments. This tool
can easily be adapted to analyse a number of 360 degree responses.
Now available:
free 360 degree appraisals form template in MSWord format
free 360 degree appraisals form template in MSExcel format
introduction of 360 degree appraisals
Here is a simple guide for introducing 360 degree appraisals into an organization (and any other
management system for that matter):
• Choose/design a system (or system provider), ie., research and investigate your options
(other local or same-sector companies using 360 already are a helpful reference point, or
your trade association HR group, or a specialist HR advisory body such as CIPD in the UK if
you are a member).
• When you've decided on a system, pilot it with a few people to make sure it does what you
expect. (It's best to establish some simple parameters or KPI's by which you can make this
assessment, rather than basing success on instinct or subjective views.)
• When satisfied with the system, launch it via a seminar or workshop, preferably including
role-plays and/or practical demonstration.

• Support the implementation with ongoing training, (include an overview in your induction
training as well), a written process guide/booklet, and also publish process and standards
on your intranet if you have one.
• Establish review and monitoring responsibility.
• Ensure any 360 degree appraisal system system is applied from top down, not bottom up,
so everyone can see that the CEO is happy to undertake what he/she expects all the other
staff to do. (As with anything else, if the CEO and board agrees to undertake it first, the
system will have much stronger take-up and credibility.)
360 degree appraisal form design - template guidelines
Job descriptions are also a useful starting point for (but by no means the full extent of)
establishing feedback criteria, as are customer/staff survey findings in which
expectations/needs/priorities of appraisee performance are indicated or implied.
A 360 degree appraisal template typically contains these column headings or fields, also shwon in
the template example below:
• Key skill/capability type (eg communications, planning, reporting, creativity and problem
solving, etc - whatever the relevant key skills and capabilities are for the role in question).
• Skill component/element (eg 'active listening and understanding' [within a
'communications' key skill], or 'generates ideas/options' [within a 'creativity/problem
solving' key skill]). The number of elements per key skill varies - for some key skills there
could be just one element; for others there could be five or six, which I'd recommend be
the maximum. Break down the key skill if there are more than six elements - big lists and
groups are less easy to work with.
• question number (purely for reference and ease of analysis)
• specific feedback question (relating to skill component, eg does the person take care to
listen and understand properly when you/others are speaking to him/her? [for the active
listening skill])
• tick-box or grade box (ideally a,b,c,d or excellent, good, not good, poor, or rate out of 5 or
10 - N.B. clarification and definitions of ratings system to participants and respondents is
crucial, especially if analysing or comparing results within a group, when obviously
consistency of interpretation of scoring is important)
360 degree feedback form template example
A typical 360 degree feedback form template would look like this. This template allows a mixture
of key skills comprising one, two, three, four, and up to six elements. The number of elements
per key skill/capability would vary of course, so if necessary adjust the size of the boxes in the
first column accordingly to accommodate more or less elements. See the notes directly above for

more explanation about the purpose of each column and heading, and the feedback scoring
method.
Feedback Form headings and instructions: appraisee name, date, feedback respondent
name, position (if applicable) plus local instructions and guidelines for completion, etc.
key skill/capability skill/capability question
feedback question feedback
area element number
1
2
3
4
5
6
7
8
9
10
Now available:
free 360 degree appraisals form template in MSWord format
free 360 degree appraisals form template in MSExcel format

You can see from this that the process of designing the feedback document (essentially a
questionnaire) is to build it from the role's key skill areas, break down these into elements, and
measure each via carefully worded questions, which the respondents answer and thereby grade
the performance - ie., give feedback - in respect of the person in question.
The question as to anonymity of respondents is up to you. A grown-up organization with grown-
up people should be able to cope with, and derive more benefit from, operating the process
transparently - but you need to decide this. Some people are happier giving feedback
anonymously. And some people are not able to deal particularly well with criticism from a named
person. For more information and guidance about handling and explaining this particular aspect
refer to the Johari Window model - it's a powerful and helpful concept to use alongside the 360
degree feedback/appraisal process.
As mentioned above, workshops are a good way to devise these questionnaires, especially the
questions to assess each skill or behavioural element.
Analysis of group results is much easier if you use a numerical rating system. The free training
needs analysis spreadsheet tool can easily be adapted for analysis of 360 degree feedback results,
which can then feed into the analysis of training needs. This training needs analysis tool is also
available in pdf format.
tips on completing your own self- assessment appraisal
form, and preparing for your appraisal
Be as truthful as you can without exposing yourself unnecessarily. Obviously if your company and/
or boss does not have a positive and fair approach be careful not to create vulnerabilities for
yourself.
Always be positive, never negative - don't complain, don't point out problems, avoid making
personal attacks on anyone or their abilities. If there are problems express them as opportunities
to develop or improve, an if possible suggest or recommend how these improvements can be
made.
Ask for help and training and coaching and development in areas that you believe will improve
your productivity and value to the organization.
Use the list or skill categories on the appraisal form to assess your capabilities and behaviours one
by one - be specific, objective and be able to reference examples and evidence. This is an
important area for the appraisal meeting itself so think about it and if necessary ask others for
feedback to help you gather examples and form a reliable view of your competence in each
category listed. If the appraisal for does not have a list of skills and behaviours create your own
(use your job description for a basis).

Assess your performance for the appraisal period (normally the past year) in each of your areas of
responsibility; if there are no specific responsibilities or objectives brought forward from your
previous appraisal or on-going meetings with your manager again use your job description as a
basis for assessing your performance, competence and achievements.
Identify objectives for yourself for the next year. These should be related to your current job
responsibilities and your intended personal development, and be a mixture of short, medium and
long-term aims (ie, days or weeks, months, and a year or more). Attach actions and measurable
outputs to these aims and objectives -this is a commitment to change and improve which
demonstrates a very responsible and mature attitude.
If your aims and actions require training or coaching or other support then state this, but do not
assume you have a right to receive it - these things cost money and your manager may not be
able to commit to them without seeking higher approval.
Think about and state your longer-term aspirations - qualifications and learning, career
development, and life issues if relevant.
Seek responsibility, work, and tasks within and beyond your normal role. Extra work and
responsibility, and achieving higher things develop people and increase productivity for and
contribution to the organization.
Always seek opportunities to help and support others, including your boss.
Always look upon reward as an economic result of your productivity. You have no 'right' to reward
or increase in reward, and reward is not driven by comparisons with what others receive. Reward,
and particularly increase in reward, results from effort and contribution to organizational
performance. As such, if you want higher reward, seek first the opportunity to contribute more.
appraisals timing with pay reviews, performance awards,
and training planning
Some people advocate separating appraisals from pay review, however this does not make sense
in organizations which require staff to be focused on their contribution to organizational
performance, especially where there are clear accountabilities and measures (which in my view
should apply in all organizations).
Organizations rightly or wrongly are geared to annual performance, and the achievement of a
trading plan. This cascades to departments, teams and individuals, so it makes sense to assess
people over a time period that fits with what the organization is working to. Put another way, it's
not easy to appraise someone on their year's performance half way through the year.
Transparency and accountability are prerequisites for proper assessment and appraisals.

Arguably 'best practice' is to schedule appraisals close to trading year-end, when year-end results
and full year performance - for individuals and departments and organizations - can reliably be
predicted. By holding appraisals at this time, and staff knowing that appraisals are focused on this
trading period, people's thoughts and efforts can be concentrated on their contribution towards
the organization's annual trading plan, which is a main appraisals driver and output (as well as
individual development of course). Holding appraisals after year-end means that people start the
year without formal agreed objectives, and also creates bigger delays for financial and payroll
departments in their task to process pay awards and adjustments.
Departmental, team and individual objectives provide the context for the appraisal, linking clearly
to performance bonus and performance-based pay awards, the rationale for which needs to be
transparent and published prior to the start of the year to which they relate, for the full benefit
and effect on staff effort to be realised.
Pay review would also coincide with the trading year, which makes sense from the planning and
budgeting perspective. The business is in a position to know by the close of the final quarter what
the overall pay review position is because the rationale has already been (it jolly well should have
been) established and year-end financials can be predicted. Moreover the next year's trading plan
(at least in outline) is established, which gives another useful context for appraising people,
especially those (most staff hopefully) who have contributed to the planning process (ie,
committed as to what they can do for the coming year, targets, budgets, staffing levels, priorities,
objectives, etc).
The appraising managers can therefore go into appraisals fully briefed and prepared to discuss
and explain the organization's overview results and financials to the appraisees. And the
appraisees can see results and think in terms of their full year performance and contribution to
corporate results, plus what they plan for next year, which provides the basis of the aims and
objectives to be reviewed through the coming year and at the next year's appraisal.
other guidelines for organizational appraisals planning
Other than for directors, complex or difficult appraisals, appraisal meetings should not be 3 hr
marathon sessions - this daft situation happens when boss and subordinate never sit down
together one-to-one other than for the annual appraisal. If you only talk properly with someone
once a year no wonder it takes all afternoon...
Boss and subordinate should ideally sit down one-to-one monthly (or at worse, quarterly, for the
more mature, self-sufficient people), to review activity, ideas, performance, progress, etc., which
makes the annual appraisal really easy when it comes around, and manageable in an hour or 90
mins max.
Use of a good appraisal form including self-assessment elements is essential for well organised
appraisals. See the template above and the free appraisal form sample.

Ensure that appraisers and appraisees understand that they must prepare in advance or you're
looking at 3 hour marathons again.
Training for appraisers and appraisees on how to use the appraisals process properly is very
helpful obviously.
pay reviews and awards
If you want to be regarded as a caring and ethical organization, it's also helpful for the
organization (board) to agree a basic across-the-board inflationary salary increase close to year
end and announce this - everyone gets this. This can be based on a collection of factors, decided
by the board, typically: inflation, the organization's financial position, demographics and
competitor market forces on salary levels.
Individuals can then receive an additional increase on top of this according to criteria agreed
before the start of the year (at their last appraisal) based on performance, achievement of
targets, job-grade advancement, qualifications attained, training aims achieved, and any other
performance levers that it is sensible, fair and practicable to incentivise.
The rationale for these individual awards must be established and budgeted for by the board,
circulated, and explained to all staff via managers.
Whilst not always easy or practicable to design and implement, arguably the best collective
annual pay increase mechanism is one that effectively rewards everyone directly and
transparently for corporate performance, ie, 'profit share' in spirit, based on the whole
organization and a business unit/department to which they relate, plus an individual performance-
linked award based on the sort of levers mentioned above. It's about people believing that they
are all part of the group effort, pulling together, and all enjoying a share of the success. Profit
share deals just for directors are rightly regarded by most staff as elitist, exclusive, and divisive. If
you want your people to give you 100%, include them in as many reward schemes as you can.
appraisals and training planning
Where appraisals coincide with year-end, training department must not rely exclusively on
appraisals data for training planning (the data arrives too late to be used for training planning for
the next year quarter 1 and probably quarter 2).
Training planning must work from data (based on audits, analyses, manager inputs,
questionnaires, market and legislative drivers, etc) gathered/received earlier during the year.
Training planning by its nature is a rolling activity and thought needs to be given to how best to
manage the data-gathering and analysis (including the vital details from staff appraisals), training
planning activity, and integrating the costs and budgeting within the corporate trading planning
process.

probationary review elements
A new employee is often subject to a probationary period - normally three months although
probationary periods vary from a few weeks to a year. Probation must have a strong link to
induction training. Probationers need to be supported properly or the chances of the new
employee struggling or failing will increase. The nature and process of probationary reviews
depend on local methods and policies, however the elements of the review process (and any
documentation or system used) will commonly be:
• name position department etc.
• dates - commencement and review
• basis of review - clear explanation of what constitutes a successful outcome, linked to
consequences of success and failure, according to probationary policies
• agreed activities and aims for probationary period
• clear and transparent quantifiable measures for each aim/activity - for acceptable
probationary review, and for ultimate job performance standard if different (aims must be
SMART - specific, measurable, agreed, realistic, time-bound - aims and activities should
logically reflect and represent the core skills, knowledge, behaviour an learning necessary
for the probationers job function)
• agreed support, training and resources for aims/activities
• names and contact details for mentors, trainers, helpers for each activity
• self-assessment section for each aim/activity
• trainer/supervisor assessment of each aim/activity
• probationary review comments and agreed future actions, per aim/activity
• overall review summary, comment and agree status/actions
• signatures and dates of reviewer and probationer
See the SMART task delegation review sheet, which is helpful for agreeing, recording and
measuring aims.
See also the skill set assessment form, which can also help the probationary review process.

The use this material is free provided copyright (Alan Chapman 1995-2006) is acknowledged
and reference or link is made to the www.businessballs.com website. This material may not
be sold, or published in any form. Disclaimer: Reliance on information, material, advice, or
other linked or recommended resources, received from Alan Chapman,
360° Feedback is a proven method of helping individuals review their
performance through the eyes of their working colleagues.
The individual first completes a self assessment, rating themselves over a serious of specific
behaviours. They then select a number of working colleagues and categorise as e.g. Manager,
Colleague, Team Member. Each selected person then assesses the individual for their current
performance under the same series of behaviours using a simple rating scale mechanism.
The feedback is then summarised and collated for the individual as a series of reports. Each report is
designed to emphasise a different aspect of the feedback e.g. Strengths, Development Areas, Opinion
Differences.
Once the individual has received the report they are in a position to identify which behaviours are seen
as in need of improvement and to choose appropriate development actions.
360 Feedback is often used as a support aid for management development training. Managers can
use the 360 report to focus on areas of the course which have been highlighted by colleagues.
360° Feedback is also highly effective as a self development tool as it provides managers with key
information which they would otherwise find hard to obtain.
RMS have been assisting organisations to run their 360° projects since 1990 and have experience in
many projects ranging in scope from just a few key staff to many thousands within a global
corporation.
Introduction

Top
Introduction
What is 360{degrees} appraisal?
What is it for?
What method should be...
Jennifer King outlines what it is and how it should be used so you won't What are the benefits...
How can 360{degrees} appraisal...
be aimlessly turning in circles
Conclusion
References
Do you know what your colleagues and patients think of you at work?
Do you see yourself as others see you? These are questions that many of
us may prefer not to ask in case we dislike the answers. But if you are a doctor you now have to provide
evidence about your working relationships with patients and colleagues to comply with the GMC's standards
on good medical practice. "360° appraisal" is suggested as one of the sources of such evidence, which will
eventually form part of the portfolio required for GMC revalidation.1 However, many doctors are unfamiliar
with 360° appraisal, and, although some doctors and departments have responded with keen interest, many
are confused and sceptical about what it is and the value of the feedback it provides.
What is 360° appraisal?
The term originated in the
commercial sector, and
refers to "full circle"
feedback from bosses,
peers, and those junior
to you. It is sometimes
referred to as "multi-
source feedback." An
individual seeks feedback
about his or her
performance at work from

a variety of colleagues
with the aim of using it to
highlight areas of strength
and those in need of
improvement. The method
evolved as the limitations
of the more traditional
"top down" appraisal
became apparent—
namely, that it was
perceived as unfair,
biased, limited to one
person's perspective
(usually the boss), and
often demotivating. 360°
appraisal had the potential
to overcome these
problems, and over the
past decade it has been
used extensively in
industry and introduced in
some general practices
and hospital departments.
What is it for?
360° appraisal can be
used for two broad
purposes—to make
decisions about pay and
promotion (performance
management) and to
determine how people
are developing their skills
and competencies.
Because of the potential
for bias, most
organisations no longer
use it for performance
management and

recognise it as a
developmental tool. It is
designed to look at the
behaviours (or
competencies) that are
key to a job including
teamwork,
communication,
managing others, and
interpersonal skills. As the
business culture has
moved towards looking
not only at what people do
but how they do it, 360°
appraisal provides a way
of measuring relevant
behaviours. It generally
does not include many
items on technical or
clinical skills as there are
other ways of measuring
these.
What method should be used?
Methods in the health
service have varied. They
include open ended,
unstructured interview
questions (such as "Tell
me three things that Dr
Smith does well and
three things he needs
to improve"), statements
with a simple rating scale
(such as "Is enthusiastic
about teaching" rated
from 1 to 5), and
structured questionnaires
based on items from focus

groups with general
practitioners or
consultants about what
they (and other healthcare
staff) consider to be
indicators of good
performance.
The most widely
publicised tool is an
American questionnaire
developed by Ramsey and
colleagues,2 who
concluded that reliable
ratings can be obtained
with 11 responses.
Another view is that there
should be at least five
people providing ratings,
to ensure some
anonymity.3 Ramsey et al
also reported that the
method of selecting
colleagues, and the
relationship between the
person doing the rating
and the person being
rated, do not substantially
bias results. However,
anonymity is crucial to the
validity of these methods.1
Typically, a 360°
feedback questionnaire
would include key
categories (such as
communication,
teamwork, etc) with five
or six specific behaviours
within each one.
Instruments will probably
be developed that
specifically reflect the
performance criteria in
"Good Medical Practice,"
but, as yet, there is no

standard or universal
method, and a variety of
tools are being tried in the
NHS.
Most questionnaires tend
to ask how well a
particular behaviour is
performed and how
important it is. In this way
it is possible to prioritise
the areas which need
developing. The
behaviours rated low on
performance but high on
importance are clearly of
higher priority than areas
considered less important
by the person doing the
rating. If the instrument
includes self rating, then
people being appraised
can see how closely their
views match with the
perceptions of others.
This is especially valuable
for team development as
different healthcare
professionals have
different views on what is
important.
When this method was
first introduced into
primary health general
practitioners asked staff
and patients to rate the
practice rather than
individual doctors. This
method was adopted by
several practices and
primary care trusts, which
used the feedback to guide
their practice
development plans. The
more courageous began to

gather personal feedback
about individual doctors,
which contributed to team
building within practices.
Some hospital
departments have started
using 360° appraisal in
this way. They tend to be
departments with an open
and supportive culture,
whose members are
willing to learn from the
feedback and use it to
enhance team
performance. Other trusts
have introduced it as part
of individual consultant
appraisal,4 but the onus is
on the person being
appraised to gather the
data and bring it to the
appraisal to help inform
the personal development
plan produced at the end.
B
e
n
e
fi

t
s
o
f
3
6
0
°
a
p
p
r
a
i
s
a
l
•
•
•
•
•
•
•
P
it
f
a
ll
s
o
f
3
6
0
°
a

p
p
r
a
i
s
a
l
•
•
•
•
•
•
•
•
•
B
a
s
e
d
o
n

o
u
r
o
w
n
a
n
d
o
t
h
e
r
s'
e
x
p
e
ri
e
n
c
e
s
o
f
d
e
s
i
g
n
i
n
g
a
n
d
i
m
p
l
e
m
e

n
ti
n
g
a
v
a
ri
e
t
y
o
f
t
o
o
l
s,
t
h
e
f
o
ll
o
w
i
n
g
a
r
e
e
s
s
e
n
ti
a
l:
•

T
•
•
•
•
•
•
•
•
M
a
n
y
q
u
e
s
t
i
o
n
s
s

t
i
l
l
h
a
v
e
t
o
b
e
a
n
s
w
e
r
e
d
a
b
o
u
t
3
6
0
°
a
p
p
r
a
i
s
a
l
,
b

u
t
i
t
h
a
s
t
h
e
p
o
t
e
n
t
i
a
l
t
o
b
e
a
u
s
e
f
u
l
t
o
o
l
f
o
r
i
n
d
i

v
i
d
u
a
l
a
n
d
t
e
a
m
d
e
v
e
l
o
p
m
e
n
t
,
w
h
e
n
u
s
e
d
r
e
s
p
o
n
s
i

b
l
y
a
n
d
i
n
c
o
m
b
i
n
a
t
i
o
n
w
i
t
h
o
t
h
e
r
s
o
u
r
c
e
s
o
f
e
v
i

d
e
n
c
e
.
S
o
m
e
h
o
s
p
i
t
a
l
a
n
d
p
r
i
m
a
r
y
c
a
r
e
t
r
u
s
t
s
h
a
v

e
s
u
c
c
e
s
s
f
u
l
l
y
p
i
l
o
t
e
d
d
i
f
f
e
r
e
n
t
m
e
t
h
o
d
s
a
n
d
i
n

c
o
r
p
o
r
a
t
e
d
t
h
e
f
e
e
d
b
a
c
k
i
n
t
o
a
p
p
r
a
i
s
a
l
d
i
s
c
u
s
s

i
o
n
s
.
I
t
i
s
e
s
s
e
n
t
i
a
l
t
h
a
t
b
e
s
t
p
r
a
c
t
i
c
e
i
s
s
h
a
r
e
d
s

o
t
h
a
t
t
o
o
l
s
b
e
c
o
m
e
s
t
a
n
d
a
r
d
i
s
e
d
,
e
s
p
e
c
i
a
l
l
y
i
f
t

h
e
y
a
r
e
u
s
e
d
f
o
r
r
e
v
a
l
i
d
a
t
i
o
n
p
u
r
p
o
s
e
s
.
3
6
0
°
a
p

p
r
a
i
s
a
l
h
a
s
a
g
r
e
a
t
d
e
a
l
t
o
o
f
f
e
r
a
n
d
c
o
u
l
d
p
r
o
v
i
d

e
w
h
a
t
R
o
b
e
r
t
B
u
r
n
s
c
a
l
l
e
d
"
t
h
e
g
i
f
t
t
o
s
e
e
o
u
r
s
e

l
v
e
s
a
s
o
t
h
e
r
s
s
e
e
u
s
.
"
J
e
n
n
i
f
e
r
K
i
n
g
,
d
i
r
e
c
t
o
r

E
d
g
e
c
u
m
b
e
C
o
n
s
u
l
t
i
n
g
G
r
o
u
p
j
e
n
n
y
-
k
i
n
g
@
e
d
g
e
c
u
m
b
e
.
c
o
.
u
k

A
n
a
t
i
o
n
a
l
c
o
n
f
e
r
e
n
c
e
,
"
A
p
r
a
c
t
i
c
a
l
g
u
i
d
e
t
o
3
6
0
°

a
p
p
r
a
i
s
a
l
,
"
o
r
g
a
n
i
s
e
d
j
o
i
n
t
l
y
b
y
H
e
a
l
t
h
c
a
r
e
E

v
e
n
t
s
a
n
d
E
d
g
e
c
u
m
b
e
C
o
n
s
u
l
t
i
n
g
,
w
i
l
l
b
e
h
e
l
d
a
t
t
h

e
R
o
y
a
l
C
o
l
l
e
g
e
o
f
P
h
y
s
i
c
i
a
n
s
,
L
o
n
d
o
n
,
o
n
2
7
J
u
n
e

2
0
0
2
(
f
o
r
f
u
r
t
h
e
r
d
e
t
a
i
l
s
c
o
n
t
a
c
t
H
e
a
l
t
h
c
a
r
e
E
v
e
n
t

s
,
i
n
f
o
@
h
e
a
l
t
h
c
a
r
e
-
e
v
e
n
t
s
.
c
o
.
u
k
)
.
C
o
m
p
e
t
i
n
g
i
n

t
e
r
e
s
t
s
:
E
d
g
e
c
u
m
b
e
C
o
n
s
u
l
t
i
n
g
G
r
o
u
p
i
s
a
c
o
n
s
u
l

t
i
n
g
c
o
m
p
a
n
y
p
r
o
v
i
d
i
n
g
s
e
r
v
i
c
e
s
t
o
t
h
e
N
H
S
a
n
d

t
h
e
c
o
r
p
o
r
a
t
e
s
e
c
t
o
r
a
n
d
h
a
s
d
e
v
e
l
o
p
e
d
i
t
s
o
w
n

3
6
0
°
a
p
p
r
a
i
s
a
l
t
o
o
l
f
o
r
d
o
c
t
o
r
s
.