Assessment techniques and strategy

Uploaded on

This panel addresses the assessment tools that should be used, how agencies can implement an assessment strategy that results in the best hires in a timely and cost efficient manner and how agencies …

This panel addresses the assessment tools that should be used, how agencies can implement an assessment strategy that results in the best hires in a timely and cost efficient manner and how agencies can measure and report on the speed and quality of their hiring processes.

More in: Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. The Path to Improving and Measuring Hiring Results
  • 2. Today’s Presenters Beverly Dugan, Ph.D. Vice President, HumRRO Mike Heil, Ph.D. Assistant Vice President, Aon Hewitt Elaine Pulakos, Ph.D. Chief Operating Officer, PDRI2
  • 3. Agenda • Federal Hiring Reform • How Can Objective Assessments Help Meet These Requirements? • Use of Objective Assessments in the Hiring Process • Selecting or Developing Effective Assessments • Evaluating Hiring Process and Outcomes • Use of Assessments in Federal Hiring • Questions and Answers3
  • 4. Federal Hiring Reform4
  • 5. Federal Government Hiring • Unprecedented numbers of applicants • Current systems too confusing and time consuming – Baseline for hiring is 130 days • Technology helps efficiency but not quality • Ineffective hiring assessments are “weakest link” – Do not effectively screen and reduce applicant pools – Do not identify best qualified – Unqualified candidates getting through • Partnership for Public Service, 20105
  • 6. President Obama’s Hiring Reform Memorandum • Improve hiring process and quality of hires – Eliminate KSA essays – Reduce time to hire mission-critical and commonly filled positions – Measure speed and effectiveness of hiring process – Measure quality of hires – Analyze causes of hiring problems – Implement actions to reduce problems Quality, Speed, and Measurement are the Key Requirements of Hiring Reform6
  • 7. How Can Objective Assessments Help Meet These Requirements?7
  • 8. Why Not Use Self-Report Information? Applicant Prepared Narratives Multiple Choice Questions Preparing written documents Which response best describes your experience preparing written documents? In my previous jobs, I have had a great deal of experience preparing written documents. I 1. No experience. served as a research assistant. My primary job duty was to conduct literature reviews and 2. In training situations but not on a job. write the results of these. Oftentimes, there were 100 or more research articles that I had 3. Under supervision, prepared documents in to review, some of which contained which the content was determined by others. contradictory findings. I have to look through 4. Independently prepared documents in which the articles for common findings and offer conclusions were based on my analyses. explanations base don my analyses as to why different studies may have obtained what 5. Independently prepared complex documents appeared to be contradictory results. This that integrated information from many required collecting a large amount of sources to decide on operations or policy. information, organizing it in a …. Narratives no longer permitted as first step in hiring process8
  • 9. Why Not Use Multiple Choice Self-Reports? • Often Inaccurate • Time Consuming – Some are legends in their – Takes a lot of work to develop own minds good items – Others underestimate – Customized for each vacancy themselves – Often yields unwieldy item banks • Manager Complaints • Can Disadvantage Applicants Who – Do not sufficiently reduce applicant pools to reasonable – Lack access to experience number to interview – Truthfully report lack of – Unqualified applicants on certs experience – Qualified applicants not on certs9
  • 10. Why Not Use Resumes? • Impractical when there are large numbers of applicants • Even if resumes could be screened efficiently (e.g., automated tool), they are not effective hiring tools – Also rely on self-report information – Information not standardized across applicants so difficult to assess10
  • 11. Why Use Objective Assessments? Objective Assessments: Systematic, accurate measurement of a job-relevant skill, knowledge area, or other competency Tell Me vs. Show Me11
  • 12. Use of Objective Assessments in the Hiring Process12
  • 13. Placement of Assessments in Hiring Process Job Posting Application Remote Assessments Automated and Unproctored On-site Proctored Verification Assessments and Interview Assessment Great Hiring Decision13
  • 14. Assessment Administration Options 1. Application-based • Slow, resource-intensive, recruitments, proctored outdated, poor candidate paper-and-pencil experience assessments per job opening 2. Resume-based • Fast, but still resource- recruitments, proctored web- intensive, and poor based assessments on a candidate experience continuous basis 3. Web resume-based • Very fast, low-cost, low- recruitments, unproctored overhead, better candidate web-based assessments on experience a continuous basis14
  • 15. Unproctored Testing Potential Risks Potential Benefits • Cheating • Reach larger and more • Applicant identity verification diverse applicant pool • Test content exposure • Reduced travel cost for • Unstandardized testing candidates environments • Reduced staffing cost for • End-user technical issues Assessment Center • Reduced recruitment Cheating, content exposure, and time/cost applicant identity all mitigated with security measures • “Cutting Edge” Image15
  • 16. Selecting or Developing Effective Assessments16
  • 17. Decide What Needs to be Assessed • Competencies – The knowledge, skills, abilities, and other attributes required for effective performance on the job • Competency Model – Collection of required competencies in an occupation or organization17
  • 18. Example Competencies Critical Customer Productive Auditing Interpersonal Communication Thinking Service Engagement Reading Writing Reasoning Self Management Presenting Information Orally Math Reasoning Flexibility Persuading and Influencing Problem Solving Stress Tolerance Decision Making Planning/Evaluate Strategic Thinking18
  • 19. Identify the Most Important Competencies Administer validation surveyto job incumbents to identify important competencies19
  • 20. Select Assessments to Measure Competencies Types of Competency Assessments Most Important Competencies Situational Work Technical Cognitive Work Styles Judgment Writing Simulations Proficiency Reading Reasoning Self Management Interpersonal Flexibility Stress Tolerance Problem Solving Decision Making Writing Auditing Planning20
  • 21. Cognitive Assessments Measure mental competencies, such as critical thinking, math, reading, problem solving To protect against publishing of items, computer selects different items for each applicant Virtually no two applicants receive same items21 21
  • 22. Work Styles Assessments Provides assessment of Applicants choose soft skill competencies, Difficult to cheat which item is most such as self management, because scoring is descriptive of them initiative, flexibility, not obvious teamwork, stress tolerance, and others22 22
  • 23. Situational Judgment Assessments Realistic problems like encountered on job Applicants select most and least effective way to handle situation Difficult to cheat because scoring is not obvious Positive applicant reactions23
  • 24. Writing Assessments Compose free text response to question Please describe why you are a good fit for this role, including relevant background experiences, and Response automatically what you feel is your greatest strength. scored on clarity, grammar, persuasion, content, and others24
  • 25. Work Simulation Assessments Realistic problems like encountered on job Can assess many competencies Applicant reactions uniformly positive due to high job relevance25
  • 26. Technical Proficiency Assessments Information Technology Assessments MS Office Assessments26
  • 27. Job Interview Assessments Job interviews are also assessments For reliable and valid assessment: One set of competency questions should be used for each vacancy and asked of all candidates All candidates should be evaluated against standard rating criteria27
  • 28. Evaluating Hiring Process and Outcomes28
  • 29. Getting Started • Define measures – Hiring reform requirements – Agency strategic goals – Most critical impact areas (reduce costs, reduce turnover, etc.) • Determine measurement strategy for stable, credible results – Collect baseline measures before changing your process – Collect measures at regular intervals during and after implementation29
  • 30. Hiring Process Evaluation • Process measures answer the question: How well is the new hiring process working? • E2E metrics focus on speed and effectiveness of hiring – % employees hired within 80 days – % applicants given information on status of application – % new employees reporting satisfaction with hiring process – % new employees reporting regular communication during hiring process • Other example hiring process metrics – Total time HR devotes to hiring process – Total time manager devotes to hiring process – Manager satisfaction with hiring process30
  • 31. Improving the Hiring Process • Technology is key “Many problems agencies face are information technology- related,” said Tim McManus, vice president of education and outreach at the nonprofit Partnership for Public Service. For example, agencies have in place automated systems to sort through knowledge, skills and abilities essays, but have not yet developed assessments based on shorter questionnaires and specific positions and qualifications. “How do you assess in a way thats automated?” he said. “Its hugely important in this conversation because of the volume of applicants agencies get -- hundreds and even thousands of applications for a single position.” Justice, Treasury and Veterans Affairs departments all identified delays caused by limited or antiquated IT systems. Excerpt from: Hiring reform deadline creates some headaches for agencies By Emily Long, September 16, 201031
  • 32. How Technology Improves the Hiring Process • Automated tools enable applicants to apply anywhere, anytime • Easy, efficient process attracts a larger number of qualified applicants • Real time, immediate category scores – Not Qualified, Qualified, or Best Qualified32
  • 33. Caveat from MSPB • “Plan carefully when implementing automated tools designed to support the hiring process. Do not assume that technology alone will fix already- broken processes. For instance, automating an assessment tool that is a low predictor of job success will not improve the tool’s predictive ability.” Excerpted from: MSPB report “Reforming federal hiring – beyond faster and cheaper” – September 1, 200633
  • 34. Example Quality of Hire Measures • Productivity • Cost savings • Retention • Time to promotion • Training time and costs • Manager satisfaction34
  • 35. Example Results from Private Sector35
  • 36. Increased Efficiency and Quality 34% more reach quality goals Call center agents $4 million realized from improved Logistics staff work efficiency 18 days of additional productivity At home agents per employee annually EFFICIENCY 35% reduced average handle time Call center agents AND QUALITY OUTCOMES 43% more likely to pass exam Claims adjustors 8% more calls completed Insurance agents 40% reduction in handle time Call center agents 18% more effective and 28% more Claims adjustors likely to hire again36
  • 37. Reduced Costs 46% reduced turnover Nurses $2 million saved in reduced repeat Technicians service calls 23% reduced turnover Call Center Agents COST 33% reduced turnover Reservations Agents REDUCTION OUTCOMES 83% reduced turnover Call Center Agents 78% million in reduced shrinkage Store Employees 50% reduced turnover Production Workers 63% reduced turnover Call Center Agents37
  • 38. Improved Leadership 300% higher performance ratings Frontline Managers 57% more likely to be promoted Branch Managers 8% higher sales and profit Territory Managers LEADERSHIP 59% higher employee engagement Branch Managers EFFECTIVENESS OUTCOMES 30% more likely to be promoted Senior Managers38
  • 39. Effective Hiring Outcomes Recap Process Hiring System Effectiveness and Speed Technology Improvements Time to Hire Applicant Process Satisfaction Improvements Applicant Communication Objective Assessments Quality of Hire Competencies Effective Work Productivity Hires with job-related Behaviors Knowledge abilities, skills Turnover Skills and traits Fit Cost Savings Work Styles Engagement Show Me vs. Manager Tell Me Satisfaction39
  • 40. Use of Objective Assessments in Federal Hiring40
  • 41. Federal Agencies Using Objective Assessments • ATF • DEA • Department of Commerce • Department of State • DoD – uniformed services • DHS - CBP • FAA • FBI • IRS • Intelligence Community • OPM • Postal Service • TSA • Veterans Benefit Administration41
  • 42. Federal Agencies Using Unproctored Assessment • Intelligence Community – Assessments include writing, foreign language, work styles, computer skills, English skills, math, proofreading, typing speed and accuracy, among others • OPM – Assessments include reading, reasoning, math reasoning, work styles, and situational judgment • U.S. Postal Service – Assessments including situational judgment and personal characteristics42
  • 43. Q&AQuestions
  • 44. Why Objective Assessments “Aren’t for Me” • I’m only hiring one person for one position – But …agency may be hiring hundreds or thousands – Competency assessments (e.g., writing) useful across jobs – Can get thousands of applicants for one job – how do you find the ONE? • Too complex, time consuming, expensive – Quality assessments available from OPM – Commercial assessments available – Usually easier to implement competency assessments than self report experience items44
  • 45. How Does Category Rating Work? • Purpose – Increases number of qualified applicants from which to select – Provides flexibility to select without regard to “rule of three” – Preserves Veterans preference rights • Process – Evaluate minimally qualified applicants – Place all minimally qualified applicants into two or more pre- determined quality categories – Combine quality categories if fewer than three in highest category – Send names of all in highest category to hiring official – Make selections form highest quality category45
  • 46. How Does Veterans Preference Work Under Category Rating? • Applied after assessment and category placement • No longer assign points • Preference eligibles receive absolute preference – Listed ahead of non-preference eligibles in the category • Preference eligibles with compensable service-connected disability of 10% must be placed in highest category (except for scientific or professional positions GS-9 and above) • Agencies must request to pass over a preference eligible46