Patterson

Evaluating new methods &approaches for high stakes selection: Implications for policy & practice Professor Fiona Patterson Work Psychology Group & University of Cambridge University of Athens, 2011

Overview• The selection process & best practice• Case studies of new selection methods & approaches in high stakes selection Case Study 1: Selecting Doctors Case Study 2: Selecting Private Bankers• Implications for policy & practice

The Selection Process Create Job competency Attract pool of Analysis model applicants Tasks, roles, Create person Choose selection Selectionresponsibilities specification methods Knowledge, Interviews, Select out skills, abilities, Simulations, unsuitableRequired level characteristics Psychometric tests applicantsof competence Reliability, validity, Identify utility, fairness Select in (rank) selection suitable criteria applicants Validate selection Evaluate Make decisions candidate selection reactions decisions

Case Study 1.Selecting doctors in the UK NationalHealth Service (NHS)

Organisational context• High profile, strong public interest• Large applicant numbers - security risks• Legal scrutiny of selection processes• Applicants above average intelligence• Strong professional ‘Trade Union’• Independent coaching firms - sole purpose is to help applicants be successful in selection

Selection in medicine through the ages… “It isn’t an interview – just “ Fill out the an informal“Work for me, son – I application chat, sweetie. knew your father.” form for HR Just a and the job is formality.” yours, mate.” 1970 1980 1990

• 23,000 applicants for 8,000 medical school places Help!• 8,000 medical students apply for their first post• 10,000 speciality applicants• 24,000 + interviews• Weeks of offering, rejecting, cascading 1000s Consultant hours

Key questions What attributes are important to be an effective clinician? What selection methods are available to test these attributes? Given the costs, beyond some basic assessment, is a lottery the best option?

Why not use a lottery system?

Selecting doctors in the UK• 8,000 applicants per year for 3,250 training posts in a centrally coordinated recruitment office – job analysis – design of situational judgement tests (SJTs) – validation studies

Job analysisThree independent studies (Patterson et al, 2000):• interviews with doctors• observations studies• interviews with patients1) Empathy & sensitivity (sensitive to patients emotions & feelings)2) Communication skills (active listening, clarity of explanation)3) Problem-solving (identifies root cause & decision-making)4) Professional integrity (respect, vocational enthusiasm)5) Coping with pressure (calm under pressure)6) Clinical expertise (clinical process awareness, identifying options)

Job Role “…establish immediate empathic rapport…define nature, history of problems, the aetiology, patient ideas, concerns, expectations, effects of problems….consider other continuing problems/at-risk factors…reach shared understanding of problems with patient….choose appropriate action for each problem…use time/resources appropriately” (Pendleton et al, 1984)• Role requires rapid decision-making in an interpersonal environment, involving complex & significant cognitive/ non-cognitive demands of the individual

Using Situational Judgement Tests(SJTs) to select doctors

What’s a Situational Judgement Test?• Situational Judgement Tests (SJTs) are a measurement method designed to assess judgement in work-relevant situations: – Present challenging situations likely to be encountered at work – Make judgements about possible responses – Scored against expert responses• Validity evidence is well-established

Example SJT itemYou are reviewing a routine drug chart for a patient withrheumatoid arthritis during an overnight shift. You notice thatyour consultant has inappropriately prescribed methotrexate7.5mg daily instead of weekly.Rank in order the following actions in response to this situation (1= Mostappropriate; 5= Least appropriate)A Ask the nurses if the consultant has made any other drug errors recentlyB Correct the prescription to 7.5mg weeklyC Leave the prescription unchanged until the consultant ward round the following morningD Phone the consultant at home to ask about changing the prescriptionE Inform the patient of the error

Validation studies• Cost-effective to develop, administer & score• Validation studies show good predictive validity• Favourable applicant reactions• Number failures reduced from 8% to less than 1%• Significant savings in human cost (to doctors & patients)• Significant savings in financial cost

Case Study 2.Selecting Private Bankers

Resolving Tensionsbetween Growing Profit &being a Trusted Advisorfor Private Bankers at RBS

Private Bankers

Key questions What attributes are important to be an effective private banker? What are the job role requirements? What selection methods are available to test these attributes?

Job analysis of Private Bankers• Provide investment services to customers across a number of services including investment & portfolio management; tax, estate planning & stock broking• Capabilities include: – Build excellent client relationships & looks after the client’s long-term interests – Generating income – Act as a part of a team – Follow regulations & manage risk

Private Banker Tension / PotentialCapability Dynamics Conflict GROWING Positive interaction PROFIT BEING A TRUSTED BUILDING CLIENT ADVISOR RELATIONSHIPS ORGANISATION, TEAM COMPLIANCE & INVOLVEMENT MANAGING RISK

Using SituationalJudgement Tests (SJTs) toselect bankers

Example Situational Judgement Test itemYou have a meeting with a potential new client scheduled for later today. Hisname is Ioannis, he is 49 years old and a Director of an company calledComputers-R-Us. Computers-R-Us banks with your company and Ioannisholds personal banking accounts with Barclays (a competitor). Ioannis isplanning to sell his business and is likely to have substantial money to investas a result.Rank the importance of information you wish to elicit during the meeting (1= Mostappropriate and 5 = Least appropriate)A An understanding of how much income Ioannis has – now and in the future both in terms of what he earns and spendsB Details of the intended sale of Computers-R-US – when this will take place, how much it is likely to be sold for, who are the key decision makers etcC What types of personal banking services Ioannis already uses and any areas where your bank could enhance this serviceD Details about Ioannis’s personal/family situationE Any interests (business or otherwise) Ioannis has outside of Computers-R-Us.

Implications• Job analysis results used to explore the dynamics between capabilities - which may help make better selection decisions• Understanding how someone manages the tensions in their role• Gives applicants a realistic job preview• Evaluation shows early signs of improved predictive validity & added value• Challenge – there is reluctance from some stakeholders to expose the tensions in the role requirements

Implications for policy & practice• Importance of bespoke job analysis in high stakes selection• Selection methods in high stakes settings must reflect the capability dynamics & tensions in the job role• SJTs useful & valid method for assessing important professional attributes• Strong business case for the added value of bespoke evidence-based measures, although persuading client groups is often challenging• Evidence-based approach is the key to stakeholder buy-in where results are tailored to the organisational context

Thank you fcp27@cam.ac.ukf.patterson@workpsychologygroup.com

Generic competency frameworksBenefits?• Defines what is required of all employees and/or leaders• Seen to act as ‘glue’ that binds people together• Provides sense of what is important (values, culture)• Drives consistency in performance management, development, induction, appraisal, succession planning, etcLimitations?• Fails to identify role specific requirements that are different eg. HR consulting model, motivational qualities associated with call centre tenure• Comprehensiveness is overwhelming & difficult to use by line managers• Inhibits use of business specific priorities• Difficult to refresh – scale of change

Designing a new selection system for Private Bankers• Move to develop role specific capability frameworks: – Off-the-shelf competency frameworks may prove invalid and unreliable without situational customisation (Raelin & Cooledge,1995)• Focus on core attributes that underpin success• Describe in term of – Motivation, ability, interpersonal characteristics• Simplify to enable change to take place ‘just in time’• Build for the line, not HR / psychologists• Provide complexity where needed and to those who need it• Understand the interplay between capabilities (dynamic model)

Patterson

by Ioannis Nikolaou on Jun 24, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Patterson — Presentation Transcript