Assessment techniques and strategy

1,466 views
1,329 views

Published on

This panel addresses the assessment tools that should be used, how agencies can implement an assessment strategy that results in the best hires in a timely and cost efficient manner and how agencies can measure and report on the speed and quality of their hiring processes.

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,466
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
31
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Assessment techniques and strategy

  1. 1. The Path to Improving and Measuring Hiring Results
  2. 2. Today’s Presenters Beverly Dugan, Ph.D. Vice President, HumRRO Mike Heil, Ph.D. Assistant Vice President, Aon Hewitt Elaine Pulakos, Ph.D. Chief Operating Officer, PDRI2
  3. 3. Agenda • Federal Hiring Reform • How Can Objective Assessments Help Meet These Requirements? • Use of Objective Assessments in the Hiring Process • Selecting or Developing Effective Assessments • Evaluating Hiring Process and Outcomes • Use of Assessments in Federal Hiring • Questions and Answers3
  4. 4. Federal Hiring Reform4
  5. 5. Federal Government Hiring • Unprecedented numbers of applicants • Current systems too confusing and time consuming – Baseline for hiring is 130 days • Technology helps efficiency but not quality • Ineffective hiring assessments are “weakest link” – Do not effectively screen and reduce applicant pools – Do not identify best qualified – Unqualified candidates getting through • Partnership for Public Service, 20105
  6. 6. President Obama’s Hiring Reform Memorandum • Improve hiring process and quality of hires – Eliminate KSA essays – Reduce time to hire mission-critical and commonly filled positions – Measure speed and effectiveness of hiring process – Measure quality of hires – Analyze causes of hiring problems – Implement actions to reduce problems Quality, Speed, and Measurement are the Key Requirements of Hiring Reform6
  7. 7. How Can Objective Assessments Help Meet These Requirements?7
  8. 8. Why Not Use Self-Report Information? Applicant Prepared Narratives Multiple Choice Questions Preparing written documents Which response best describes your experience preparing written documents? In my previous jobs, I have had a great deal of experience preparing written documents. I 1. No experience. served as a research assistant. My primary job duty was to conduct literature reviews and 2. In training situations but not on a job. write the results of these. Oftentimes, there were 100 or more research articles that I had 3. Under supervision, prepared documents in to review, some of which contained which the content was determined by others. contradictory findings. I have to look through 4. Independently prepared documents in which the articles for common findings and offer conclusions were based on my analyses. explanations base don my analyses as to why different studies may have obtained what 5. Independently prepared complex documents appeared to be contradictory results. This that integrated information from many required collecting a large amount of sources to decide on operations or policy. information, organizing it in a …. Narratives no longer permitted as first step in hiring process8
  9. 9. Why Not Use Multiple Choice Self-Reports? • Often Inaccurate • Time Consuming – Some are legends in their – Takes a lot of work to develop own minds good items – Others underestimate – Customized for each vacancy themselves – Often yields unwieldy item banks • Manager Complaints • Can Disadvantage Applicants Who – Do not sufficiently reduce applicant pools to reasonable – Lack access to experience number to interview – Truthfully report lack of – Unqualified applicants on certs experience – Qualified applicants not on certs9
  10. 10. Why Not Use Resumes? • Impractical when there are large numbers of applicants • Even if resumes could be screened efficiently (e.g., automated tool), they are not effective hiring tools – Also rely on self-report information – Information not standardized across applicants so difficult to assess10
  11. 11. Why Use Objective Assessments? Objective Assessments: Systematic, accurate measurement of a job-relevant skill, knowledge area, or other competency Tell Me vs. Show Me11
  12. 12. Use of Objective Assessments in the Hiring Process12
  13. 13. Placement of Assessments in Hiring Process Job Posting Application Remote Assessments Automated and Unproctored On-site Proctored Verification Assessments and Interview Assessment Great Hiring Decision13
  14. 14. Assessment Administration Options 1. Application-based • Slow, resource-intensive, recruitments, proctored outdated, poor candidate paper-and-pencil experience assessments per job opening 2. Resume-based • Fast, but still resource- recruitments, proctored web- intensive, and poor based assessments on a candidate experience continuous basis 3. Web resume-based • Very fast, low-cost, low- recruitments, unproctored overhead, better candidate web-based assessments on experience a continuous basis14
  15. 15. Unproctored Testing Potential Risks Potential Benefits • Cheating • Reach larger and more • Applicant identity verification diverse applicant pool • Test content exposure • Reduced travel cost for • Unstandardized testing candidates environments • Reduced staffing cost for • End-user technical issues Assessment Center • Reduced recruitment Cheating, content exposure, and time/cost applicant identity all mitigated with security measures • “Cutting Edge” Image15
  16. 16. Selecting or Developing Effective Assessments16
  17. 17. Decide What Needs to be Assessed • Competencies – The knowledge, skills, abilities, and other attributes required for effective performance on the job • Competency Model – Collection of required competencies in an occupation or organization17
  18. 18. Example Competencies Critical Customer Productive Auditing Interpersonal Communication Thinking Service Engagement Reading Writing Reasoning Self Management Presenting Information Orally Math Reasoning Flexibility Persuading and Influencing Problem Solving Stress Tolerance Decision Making Planning/Evaluate Strategic Thinking18
  19. 19. Identify the Most Important Competencies Administer validation surveyto job incumbents to identify important competencies19
  20. 20. Select Assessments to Measure Competencies Types of Competency Assessments Most Important Competencies Situational Work Technical Cognitive Work Styles Judgment Writing Simulations Proficiency Reading Reasoning Self Management Interpersonal Flexibility Stress Tolerance Problem Solving Decision Making Writing Auditing Planning20
  21. 21. Cognitive Assessments Measure mental competencies, such as critical thinking, math, reading, problem solving To protect against publishing of items, computer selects different items for each applicant Virtually no two applicants receive same items21 21
  22. 22. Work Styles Assessments Provides assessment of Applicants choose soft skill competencies, Difficult to cheat which item is most such as self management, because scoring is descriptive of them initiative, flexibility, not obvious teamwork, stress tolerance, and others22 22
  23. 23. Situational Judgment Assessments Realistic problems like encountered on job Applicants select most and least effective way to handle situation Difficult to cheat because scoring is not obvious Positive applicant reactions23
  24. 24. Writing Assessments Compose free text response to question Please describe why you are a good fit for this role, including relevant background experiences, and Response automatically what you feel is your greatest strength. scored on clarity, grammar, persuasion, content, and others24
  25. 25. Work Simulation Assessments Realistic problems like encountered on job Can assess many competencies Applicant reactions uniformly positive due to high job relevance25
  26. 26. Technical Proficiency Assessments Information Technology Assessments MS Office Assessments26
  27. 27. Job Interview Assessments Job interviews are also assessments For reliable and valid assessment: One set of competency questions should be used for each vacancy and asked of all candidates All candidates should be evaluated against standard rating criteria27
  28. 28. Evaluating Hiring Process and Outcomes28
  29. 29. Getting Started • Define measures – Hiring reform requirements – Agency strategic goals – Most critical impact areas (reduce costs, reduce turnover, etc.) • Determine measurement strategy for stable, credible results – Collect baseline measures before changing your process – Collect measures at regular intervals during and after implementation29
  30. 30. Hiring Process Evaluation • Process measures answer the question: How well is the new hiring process working? • E2E metrics focus on speed and effectiveness of hiring – % employees hired within 80 days – % applicants given information on status of application – % new employees reporting satisfaction with hiring process – % new employees reporting regular communication during hiring process • Other example hiring process metrics – Total time HR devotes to hiring process – Total time manager devotes to hiring process – Manager satisfaction with hiring process30
  31. 31. Improving the Hiring Process • Technology is key “Many problems agencies face are information technology- related,” said Tim McManus, vice president of education and outreach at the nonprofit Partnership for Public Service. For example, agencies have in place automated systems to sort through knowledge, skills and abilities essays, but have not yet developed assessments based on shorter questionnaires and specific positions and qualifications. “How do you assess in a way thats automated?” he said. “Its hugely important in this conversation because of the volume of applicants agencies get -- hundreds and even thousands of applications for a single position.” Justice, Treasury and Veterans Affairs departments all identified delays caused by limited or antiquated IT systems. Excerpt from: Hiring reform deadline creates some headaches for agencies By Emily Long elong@govexec.com, September 16, 201031
  32. 32. How Technology Improves the Hiring Process • Automated tools enable applicants to apply anywhere, anytime • Easy, efficient process attracts a larger number of qualified applicants • Real time, immediate category scores – Not Qualified, Qualified, or Best Qualified32
  33. 33. Caveat from MSPB • “Plan carefully when implementing automated tools designed to support the hiring process. Do not assume that technology alone will fix already- broken processes. For instance, automating an assessment tool that is a low predictor of job success will not improve the tool’s predictive ability.” Excerpted from: MSPB report “Reforming federal hiring – beyond faster and cheaper” – September 1, 200633
  34. 34. Example Quality of Hire Measures • Productivity • Cost savings • Retention • Time to promotion • Training time and costs • Manager satisfaction34
  35. 35. Example Results from Private Sector35
  36. 36. Increased Efficiency and Quality 34% more reach quality goals Call center agents $4 million realized from improved Logistics staff work efficiency 18 days of additional productivity At home agents per employee annually EFFICIENCY 35% reduced average handle time Call center agents AND QUALITY OUTCOMES 43% more likely to pass exam Claims adjustors 8% more calls completed Insurance agents 40% reduction in handle time Call center agents 18% more effective and 28% more Claims adjustors likely to hire again36
  37. 37. Reduced Costs 46% reduced turnover Nurses $2 million saved in reduced repeat Technicians service calls 23% reduced turnover Call Center Agents COST 33% reduced turnover Reservations Agents REDUCTION OUTCOMES 83% reduced turnover Call Center Agents 78% million in reduced shrinkage Store Employees 50% reduced turnover Production Workers 63% reduced turnover Call Center Agents37
  38. 38. Improved Leadership 300% higher performance ratings Frontline Managers 57% more likely to be promoted Branch Managers 8% higher sales and profit Territory Managers LEADERSHIP 59% higher employee engagement Branch Managers EFFECTIVENESS OUTCOMES 30% more likely to be promoted Senior Managers38
  39. 39. Effective Hiring Outcomes Recap Process Hiring System Effectiveness and Speed Technology Improvements Time to Hire Applicant Process Satisfaction Improvements Applicant Communication Objective Assessments Quality of Hire Competencies Effective Work Productivity Hires with job-related Behaviors Knowledge abilities, skills Turnover Skills and traits Fit Cost Savings Work Styles Engagement Show Me vs. Manager Tell Me Satisfaction39
  40. 40. Use of Objective Assessments in Federal Hiring40
  41. 41. Federal Agencies Using Objective Assessments • ATF • DEA • Department of Commerce • Department of State • DoD – uniformed services • DHS - CBP • FAA • FBI • IRS • Intelligence Community • OPM • Postal Service • TSA • Veterans Benefit Administration41
  42. 42. Federal Agencies Using Unproctored Assessment • Intelligence Community – Assessments include writing, foreign language, work styles, computer skills, English skills, math, proofreading, typing speed and accuracy, among others • OPM – Assessments include reading, reasoning, math reasoning, work styles, and situational judgment • U.S. Postal Service – Assessments including situational judgment and personal characteristics42
  43. 43. Q&AQuestions
  44. 44. Why Objective Assessments “Aren’t for Me” • I’m only hiring one person for one position – But …agency may be hiring hundreds or thousands – Competency assessments (e.g., writing) useful across jobs – Can get thousands of applicants for one job – how do you find the ONE? • Too complex, time consuming, expensive – Quality assessments available from OPM – Commercial assessments available – Usually easier to implement competency assessments than self report experience items44
  45. 45. How Does Category Rating Work? • Purpose – Increases number of qualified applicants from which to select – Provides flexibility to select without regard to “rule of three” – Preserves Veterans preference rights • Process – Evaluate minimally qualified applicants – Place all minimally qualified applicants into two or more pre- determined quality categories – Combine quality categories if fewer than three in highest category – Send names of all in highest category to hiring official – Make selections form highest quality category45
  46. 46. How Does Veterans Preference Work Under Category Rating? • Applied after assessment and category placement • No longer assign points • Preference eligibles receive absolute preference – Listed ahead of non-preference eligibles in the category • Preference eligibles with compensable service-connected disability of 10% must be placed in highest category (except for scientific or professional positions GS-9 and above) • Agencies must request to pass over a preference eligible46

×