Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Evaluating new methods &approaches for high stakes selection:  Implications for policy & practice            Professor Fio...
Overview• The selection process & best practice• Case studies of new selection methods & approaches  in high stakes select...
The Selection Process                          Create   Job                  competency                                   ...
Case Study 1.Selecting doctors in the UK NationalHealth Service (NHS)
Organisational context• High profile, strong public interest• Large applicant numbers - security risks• Legal scrutiny of ...
Selection in medicine through the ages…      “It isn’t an                                          interview – just       ...
• 23,000  applicants for  8,000 medical  school places                          Help!• 8,000 medical  students apply  for ...
Key questions What attributes are important to be an effective clinician? What selection methods are available to test the...
Why not use a lottery system?
Selecting doctors in the UK• 8,000 applicants per year for 3,250 training posts in a  centrally coordinated recruitment of...
Job analysisThree independent studies (Patterson et al, 2000):• interviews with doctors• observations studies• interviews ...
Job Role  “…establish immediate empathic rapport…define nature,  history of problems, the aetiology, patient ideas, concer...
Using Situational Judgement Tests(SJTs) to select doctors
What’s a Situational Judgement Test?• Situational Judgement Tests (SJTs) are a measurement  method designed to assess judg...
Example SJT itemYou are reviewing a routine drug chart for a patient withrheumatoid arthritis during an overnight shift. Y...
Validation studies• Cost-effective to develop, administer & score• Validation studies show good predictive validity•   Fav...
Case Study 2.Selecting Private Bankers
Resolving Tensionsbetween Growing Profit &being a Trusted Advisorfor Private Bankers at RBS
Private Bankers
Key questions What attributes are important to be an effective private banker? What are the job role requirements? What se...
Job analysis of Private Bankers• Provide investment services to customers across a number  of services including investmen...
Private Banker                                  Tension / PotentialCapability Dynamics                             Conflic...
Using SituationalJudgement Tests (SJTs) toselect bankers
Example Situational Judgement Test itemYou have a meeting with a potential new client scheduled for later today. Hisname i...
Example Situational Judgement Test itemYou have a meeting with a potential new client scheduled for later today. Hisname i...
Implications• Job analysis results used to explore the dynamics between  capabilities - which may help make better selecti...
Implications for policy & practice• Importance of bespoke job analysis in high stakes selection• Selection methods in high...
Thank you           fcp27@cam.ac.ukf.patterson@workpsychologygroup.com
Generic competency frameworksBenefits?• Defines what is required of all employees and/or leaders• Seen to act as ‘glue’ th...
Designing a new selection system for  Private Bankers• Move to develop role specific capability frameworks:   – Off-the-sh...
Upcoming SlideShare
Loading in …5
×

Patterson

1,389 views

Published on

Fiona Patterson's presentation

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Patterson

  1. 1. Evaluating new methods &approaches for high stakes selection: Implications for policy & practice Professor Fiona Patterson Work Psychology Group & University of Cambridge University of Athens, 2011
  2. 2. Overview• The selection process & best practice• Case studies of new selection methods & approaches in high stakes selection Case Study 1: Selecting Doctors Case Study 2: Selecting Private Bankers• Implications for policy & practice
  3. 3. The Selection Process Create Job competency Attract pool of Analysis model applicants Tasks, roles, Create person Choose selection Selectionresponsibilities specification methods Knowledge, Interviews, Select out skills, abilities, Simulations, unsuitableRequired level characteristics Psychometric tests applicantsof competence Reliability, validity, Identify utility, fairness Select in (rank) selection suitable criteria applicants Validate selection Evaluate Make decisions candidate selection reactions decisions
  4. 4. Case Study 1.Selecting doctors in the UK NationalHealth Service (NHS)
  5. 5. Organisational context• High profile, strong public interest• Large applicant numbers - security risks• Legal scrutiny of selection processes• Applicants above average intelligence• Strong professional ‘Trade Union’• Independent coaching firms - sole purpose is to help applicants be successful in selection
  6. 6. Selection in medicine through the ages… “It isn’t an interview – just “ Fill out the an informal“Work for me, son – I application chat, sweetie. knew your father.” form for HR Just a and the job is formality.” yours, mate.” 1970 1980 1990
  7. 7. • 23,000 applicants for 8,000 medical school places Help!• 8,000 medical students apply for their first post• 10,000 speciality applicants• 24,000 + interviews• Weeks of offering, rejecting, cascading 1000s Consultant hours
  8. 8. Key questions What attributes are important to be an effective clinician? What selection methods are available to test these attributes? Given the costs, beyond some basic assessment, is a lottery the best option?
  9. 9. Why not use a lottery system?
  10. 10. Selecting doctors in the UK• 8,000 applicants per year for 3,250 training posts in a centrally coordinated recruitment office – job analysis – design of situational judgement tests (SJTs) – validation studies
  11. 11. Job analysisThree independent studies (Patterson et al, 2000):• interviews with doctors• observations studies• interviews with patients1) Empathy & sensitivity (sensitive to patients emotions & feelings)2) Communication skills (active listening, clarity of explanation)3) Problem-solving (identifies root cause & decision-making)4) Professional integrity (respect, vocational enthusiasm)5) Coping with pressure (calm under pressure)6) Clinical expertise (clinical process awareness, identifying options)
  12. 12. Job Role “…establish immediate empathic rapport…define nature, history of problems, the aetiology, patient ideas, concerns, expectations, effects of problems….consider other continuing problems/at-risk factors…reach shared understanding of problems with patient….choose appropriate action for each problem…use time/resources appropriately” (Pendleton et al, 1984)• Role requires rapid decision-making in an interpersonal environment, involving complex & significant cognitive/ non-cognitive demands of the individual
  13. 13. Using Situational Judgement Tests(SJTs) to select doctors
  14. 14. What’s a Situational Judgement Test?• Situational Judgement Tests (SJTs) are a measurement method designed to assess judgement in work-relevant situations: – Present challenging situations likely to be encountered at work – Make judgements about possible responses – Scored against expert responses• Validity evidence is well-established
  15. 15. Example SJT itemYou are reviewing a routine drug chart for a patient withrheumatoid arthritis during an overnight shift. You notice thatyour consultant has inappropriately prescribed methotrexate7.5mg daily instead of weekly.Rank in order the following actions in response to this situation (1= Mostappropriate; 5= Least appropriate)A Ask the nurses if the consultant has made any other drug errors recentlyB Correct the prescription to 7.5mg weeklyC Leave the prescription unchanged until the consultant ward round the following morningD Phone the consultant at home to ask about changing the prescriptionE Inform the patient of the error
  16. 16. Validation studies• Cost-effective to develop, administer & score• Validation studies show good predictive validity• Favourable applicant reactions• Number failures reduced from 8% to less than 1%• Significant savings in human cost (to doctors & patients)• Significant savings in financial cost
  17. 17. Case Study 2.Selecting Private Bankers
  18. 18. Resolving Tensionsbetween Growing Profit &being a Trusted Advisorfor Private Bankers at RBS
  19. 19. Private Bankers
  20. 20. Key questions What attributes are important to be an effective private banker? What are the job role requirements? What selection methods are available to test these attributes?
  21. 21. Job analysis of Private Bankers• Provide investment services to customers across a number of services including investment & portfolio management; tax, estate planning & stock broking• Capabilities include: – Build excellent client relationships & looks after the client’s long-term interests – Generating income – Act as a part of a team – Follow regulations & manage risk
  22. 22. Private Banker Tension / PotentialCapability Dynamics Conflict GROWING Positive interaction PROFIT BEING A TRUSTED BUILDING CLIENT ADVISOR RELATIONSHIPS ORGANISATION, TEAM COMPLIANCE & INVOLVEMENT MANAGING RISK
  23. 23. Using SituationalJudgement Tests (SJTs) toselect bankers
  24. 24. Example Situational Judgement Test itemYou have a meeting with a potential new client scheduled for later today. Hisname is Ioannis, he is 49 years old and a Director of an company calledComputers-R-Us. Computers-R-Us banks with your company and Ioannisholds personal banking accounts with Barclays (a competitor). Ioannis isplanning to sell his business and is likely to have substantial money to investas a result.Rank the importance of information you wish to elicit during the meeting (1= Mostappropriate and 5 = Least appropriate)A An understanding of how much income Ioannis has – now and in the future both in terms of what he earns and spendsB Details of the intended sale of Computers-R-US – when this will take place, how much it is likely to be sold for, who are the key decision makers etcC What types of personal banking services Ioannis already uses and any areas where your bank could enhance this serviceD Details about Ioannis’s personal/family situationE Any interests (business or otherwise) Ioannis has outside of Computers-R-Us.
  25. 25. Example Situational Judgement Test itemYou have a meeting with a potential new client scheduled for later today. Hisname is Ioannis, he is 49 years old and a Director of an company calledComputers-R-Us. Computers-R-Us banks with your company and Ioannisholds personal banking accounts with Barclays (a competitor). Ioannis isplanning to sell his business and is likely to have substantial money to investas a result.Rank the importance of information you wish to elicit during the meeting (1= Mostappropriate and 5 = Least appropriate)A An understanding of how much income Ioannis has – now and in the future both in terms of what he earns and spendsB Details of the intended sale of Computers-R-US – when this will take place, how much it is likely to be sold for, who are the key decision makers etcC What types of personal banking services Ioannis already uses and any areas where your bank could enhance this serviceD Details about Ioannis’s personal/family situationE Any interests (business or otherwise) Ioannis has outside of Computers-R-Us.
  26. 26. Implications• Job analysis results used to explore the dynamics between capabilities - which may help make better selection decisions• Understanding how someone manages the tensions in their role• Gives applicants a realistic job preview• Evaluation shows early signs of improved predictive validity & added value• Challenge – there is reluctance from some stakeholders to expose the tensions in the role requirements
  27. 27. Implications for policy & practice• Importance of bespoke job analysis in high stakes selection• Selection methods in high stakes settings must reflect the capability dynamics & tensions in the job role• SJTs useful & valid method for assessing important professional attributes• Strong business case for the added value of bespoke evidence-based measures, although persuading client groups is often challenging• Evidence-based approach is the key to stakeholder buy-in where results are tailored to the organisational context
  28. 28. Thank you fcp27@cam.ac.ukf.patterson@workpsychologygroup.com
  29. 29. Generic competency frameworksBenefits?• Defines what is required of all employees and/or leaders• Seen to act as ‘glue’ that binds people together• Provides sense of what is important (values, culture)• Drives consistency in performance management, development, induction, appraisal, succession planning, etcLimitations?• Fails to identify role specific requirements that are different eg. HR consulting model, motivational qualities associated with call centre tenure• Comprehensiveness is overwhelming & difficult to use by line managers• Inhibits use of business specific priorities• Difficult to refresh – scale of change
  30. 30. Designing a new selection system for Private Bankers• Move to develop role specific capability frameworks: – Off-the-shelf competency frameworks may prove invalid and unreliable without situational customisation (Raelin & Cooledge,1995)• Focus on core attributes that underpin success• Describe in term of – Motivation, ability, interpersonal characteristics• Simplify to enable change to take place ‘just in time’• Build for the line, not HR / psychologists• Provide complexity where needed and to those who need it• Understand the interplay between capabilities (dynamic model)

×