Show Me The Money! Choosing A Pre-Employment Assessment Partner


Published on

ERE Webinar from 1/18/12, presented by Dr. Charles Handler.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Stress that I am vendor neutral and that the contents here borrow from my friends in the industry.
  • Engage audience to discuss various meA sound foundation for prediction/ informed decision makingthods, what have they used, what do they know about? Raise hands
  • Survival requires adaptation. Lets look at today’s hiring process and see where assessment fits in
  • Show Me The Money! Choosing A Pre-Employment Assessment Partner

    1. 1. Charles A. Handler, Ph.D.January 18,
    2. 2. Today’s Objective Provide information and process to help ensure you have success with pre-employment assessment Teach you that success is an investment in understanding your needs and choosing a vendor who represents the optimal solution for meeting them
    3. 3. No vendor names will be used!
    4. 4. Agenda1. Assessment and its value prop2. Assessment market overview3. Choosing the right vendor*4. Questions
    5. 5. Part 1: Assessment & its value proposition
    6. 6. What is Assessment?Any method used to provide data/information about a candidate’s qualifications and potential for performing a job or his/her potential to perform the requirements of the job. So, it’s a predictive tool to support informed decision making! Believe it or not!- interviews, resume reviews, screening, and background checks all are considered to be assessments
    7. 7. Why does assessment work? Leverages prediction Created using a scientific method Applied systematically to create standardized process Provides data to support differentiation between candidates
    8. 8. What assessment tells us Personality Measures Experience Measures Ability Measures “What can “What have they do?” they done?” Future Performance “What do they want to do?” Job Fit: What do they want to do? Environment Fit: Where do they want to work?
    9. 9. Predictive Accuracy: AKA validity 0 value Zero Predictive Job Performance Predictor Content Accuracy Moderate useful Job Performance Predictor value Predictive Content Accuracy High Predictive Job Performance Predictor strong Accuracy Content value
    10. 10. 80-20: Top and bottom performers Test scores can differentiate between high and low performers. Roster Output Average The higher the validity, the better job the test does at Sally 421 identifying top and bottom Bill 402 performers. Erin 389 Nichole 374 Top 80% Jose 370 347 Aundra 293 Cyndi 276 Emil 254 Yuliya 237 Bottom 20 % George 229 233
    11. 11. Screening OutScreening In
    12. 12. Without assessment…. Information overload Low quality information No standardization Not always job related
    13. 13. With assessment…… Process efficiency Candidate quality Own the employee lifecycle Predict the “stuff” needed to fulfill business drivers Invest in human capital Provide legal defensibility
    14. 14. What assessment CAN do  Provide extra data points about an applicant to help support the decision making process  Provide a standardized way to gain insight into applicant qualities that aren’t always apparent  Provide a way to support quality of hire  Provide supporting evidence for trends in an applicant’s data  Help identify developmental needs from Day 1  Weed out the worst applicants while focusing on the best  Provide real ROI
    15. 15. What Assessment CANNOT do  Tell you all you need to know about an applicant  Solve all your problems  Make decisions for you  Guarantee applicants will be successful  Be 100% effective  Take the place of an expert who can see the big picture
    16. 16. Valued outcomes•Job performance•Increased sales•Tenure•Reduced absenteeism•Reduced theft •Customer service •Brand support •Leadership •Bench strength •And more!!!
    17. 17. What does value look like? Tangible outcomes o Reduced turnover o Reduced absence o More sales o Increased customer sat Less tangible outcomes o Bench strength o Developmental planning o Brand building o Org culture
    18. 18. Bottom line impact 300% $55 Million 46% $78 Million Higher Retained Reduction in in Reduced Performance Revenue Turnover Shrinkage Ratings Increased $120 Million Successful Reduced 18 Additional in Additional Graduate Time to Hire Days of Overall Hires by by 3 Weeks Productivity Sales 73% Employees $170 Million Employees 57% More 4 Times in Additional 3 Times Likely to be More Collections More Client- Promoted Reliable Revenue focused
    19. 19. Common assessment tools Matching tools Qualifications screening Biographical history data Personality tests Work values/ “fit” Cognitive tests Situational judgment tests Simulations Integrity tests Structured interviews Knowledge and skills testing Assessment centers Executive assessment
    20. 20. Screening OutScreening In
    21. 21. Part 2: The assessment market
    22. 22. Who wants to sell youassessments? Matching companies- work at the job search level, use database of assessed candidates, or assessment info Test publishers- Sell tests delivered via a simple system, do not alter content to match situation Profilers- Use same set of content in all situations, benchmark vs. existing employees Consulting firms- Work with firms to customize an approach, most often on a strategic level, often resell tests created by others Solution providers- Use technology infrastructure to customize solutions for clients, often embed assessment within a suite of other products Talent management firms- Offer assessment as an add on to other post-hire services. Non-traditional vendors- Usually based on some proprietary or revolutionary idea, often deviate from acceptable range
    23. 23. Evolution of Test Use1950-2000 Test Consulting Non-500k – 1 Million Publishers Firms Profilers Traditionals2001-2010 Solution Test Consulting Profilers Matching TM Non- Providers Publishers Firms1 – 25 TraditionalsMillion2011-2020 Solution Matching Talent Profilers25 – 50 Million Providers Management Consulting Test Non- Traditionals
    24. 24. Differentiating vendorsKey parameters of differentiation 1. Content 2. Implementation/Configuration model 3. Technology 4. Contextual products/services 5. Scalability 6. Business partnership 7. Cost
    25. 25. 1. Assessment Content Quality- Do they offer proven content that has a sound theoretical basis and data to show it works? Variety- Do they offer a suite of complimentary assessments or do they offer only one type? Flexibility- Can the content be modified, customized, adapted? Usability- Is the content user friendly, engaging? Relevance- Do they offer content that leverages specific industry experience, knowledge?
    26. 26. Market map: Content High Consulting Firms Matching Firms Talent Management Firms Solution Providers Test PublishersVariety Profilers Low High Non- Traditional Low Quality
    27. 27. Key questions- Content What are you trying to predict? Are you looking at a one test or an entire process? How unique is your situation? What is the level of the job for which assessment is to be used? Are you looking to hire for potential or for immediate ability to do the job? What type of applicant volume are you experiencing?
    28. 28. 2. Implementation/Configuration models Matching- Uses assessments to help align individuals and with jobs, relies on database of information as value proposition Profiling- Uses one set of content and benchmarks vs. existing employees to support hiring to a profile Off the shelf tests- Uses content that has been shown to work in specific situations based on general data, not dialed in to local situations Light customization- Uses technology system to align assessment content to match key parameters of a situation Full customization/Local configuration- Work within a specific context to research key parameters and create or align content to ensure high degree of relevance
    29. 29. Market map: Speed vs. Accuracy High Profilers Matching Firms Solution Providers Non- TraditionalAccuracy Low High Test Publishers Talent Management Firms Consulting Firms Low Speed
    30. 30. 3.Technology Innovation- Is technology used to do things differently?, does content leverage technology? Flexibility- Does the system allow for easy configuration?, access to all content in one place Usability- Does the system make it easy for end users to configure and make decisions? Reporting- Does the system have the ability to customize reports, are reports easy to understand? Analytics- Does the vendor support advanced analytics?
    31. 31. Market map: Technology Complex Deployment Simple Delivery Non-Traditional Talent Management Profilers Firms Solution Providers Test Publishers Consulting Firms Matching Firms • Flexibility • Integration Low • Content Innovation High Technology • Advanced Analytics
    32. 32. 4. Contextual products and services Post-hire/Talent management- links to development, performance management, training Payroll/Tax- Additional HR related data and services Recruitment/Sourcing- Services to find and vet candidates OD- Teambuilding, leadership development, organizational configuration, survey research ATS/HRIS- Other HR related technology systems
    33. 33. Market Map: Contextual Products & Services Variety of Products & Services Matching Firms Consulting Firms Non-Traditional Tests Only Profilers Solution Providers Test Publishers Talent Management Firms
    34. 34. 5. Scalability Enterprise level- Can the vendor handle the variety of jobs, the volume? Global- Does the vendor have global capacity? Content and support that works globally? Distributed- Can the vendor handle hiring operations in multiple, distributed locations account for differences between them? Speed of implementation model- Is the vendor agile? Technology model- Does the vendor have the technology needed to support large scale? Data integration- Can the vendor link your data to help you gain insight across many areas of the business?
    35. 35. Scalability Technology Resources Implementation Strategy
    36. 36. Market Map: Scalability vs. Strategic Value High True Business Partner Able to Support Enterprise Globally Solution Providers Hard Time Serving Enterprise Talent Management FirmsScalability Low Consulting Firms High Profilers Test Publishers Non-Traditional Matching Firms Helps with Basic HR Low Strategic Value
    37. 37. 6. Service level Personal service- How responsive is the vendor?, how available?, do they make an effort to really understand who you are? Flexibility- Can the vendor adapt to your needs? Creativity- Does the vendor work creatively, think outside a script?, or can they follow it if need be? Business results focus- Does the vendor tie their work directly to results, speak the language of business leaders? Bandwidth- Does the vendor have the infrastructure and resources to meet your needs?
    38. 38. Market map: Service levelProcessor Partner Matching Firms Talent Management Firms Consulting Firms Test Publishers Profilers Non-Traditional Solution ProvidersLow High Service Level
    39. 39. 7. Cost Should not necessarily be the #1 parameter in many situations More accurate prediction generally costs more Vendors have a variety of pricing models Cost includes set up/configuration, use, and ongoing program evaluation Most vendors moving away from a cost per test model
    40. 40. You get what you pay for Investment: Time & Money Results
    41. 41. Market map: Cost vs. Accuracy High Consulting Firms Profilers Solution ProvidersAccuracy Talent Management Firms Matching Firms Low High Test Publishers Non-Traditional Low Cost
    42. 42. Part 3: Choosing the right partner
    43. 43. Process used to select assessment From: Rocket-Hire/ERE/ survey 2011
    44. 44. Purchase and process Assessment is most often purchased informally Assessment purchase most often does not involve experts This often leads to a lack of optimization and failure to follow best practices
    45. 45. Think Optimization! Many vendors can offer a workable solution Not all solutions are created equal and what seems similar can deliver very different results It is worth the time and effort to carefully evaluate your options using a proven methodology that allows you to understand and balance your priorities
    46. 46. Ideal Fit Lower Value• Content: Alignment/Relevance/Q uality• Config model• Technology Fit Optimal Match ($) High Value• Contextual Services• Scalability• Business Partner• Cost/resources Compromise on 1-2 Compromise on > 2 Lower Value
    47. 47. 3 steps to success • Identify your objectives and create charter Discovery • Learn about your options, collect data Research • Question, evaluate, interact, decide Decision Making
    48. 48. Parameters anchor process Discovery Research Decision Success Making Key Vendor Parameters Content Config Model Technology Contextual svcs Scalability Business Partner Cost
    49. 49. Make this process your own Your goals and objectives determine how rigorous of a process you should undertake You can generalize from the steps provided It is most important to:  Clearly id valued outcomes  Do your homework  Collect input from experts  Evaluate as a team
    50. 50. Step 1: Discovery Goal is to fully understand parameters required to make assessment a successful business process It is essential to complete this step before evaluating any vendors Don’t let the cart drive the horse
    51. 51. Linking strategy to results drivessuccessOrganizational strategy Talent management strategy Desired business outcomes Measureable results Success of assessment as a business process
    52. 52. Create a cross functional team Champion Legal Procurement Assessment Selection Assessment Expert Team End users Stakeholders Staffing/HR
    53. 53. Create your charter Strategic Focus for the initiative Define vendor Role within hiring evaluation process process Outcome data Professional defining impact services/support Assessment Charter Resources Scope- available # jobs involved (time and $) Level of customization Technology Requirements required
    54. 54. Your charter Dictates what you need and why Dictates clearly where assessment fill fit within process and talent management strategy Clearly defines business contribution from assessment Dictates who will help evaluate options Dictates timing till ramped up Dictates budget parameters Define hiring process
    55. 55. Step 2: ResearchGoal = Use your charter and your team to ID a final set of vendors who best meet your needs
    56. 56. Vendor research process Initial research Initial data collection Initial evaluation Final vendor list
    57. 57. Initial researchGoal is to identify domain of vendors for more in- depth review  Requires cutting through the crap to figure out what each vendor really does and how they are differentiated  Helps to classify vendors and understand the tradeoffs required by their approach  Collect the info needed to understand where they land on all key parameters
    58. 58. Who wants to sell youassessments? Matching companies- work at the job search level, use database of assessed candidates, or assessment info Test publishers- Sell tests delivered via a simple system, do not alter content to match situation Profilers- Use same set of content in all situations, benchmark vs. existing employees Consulting firms- Work with firms to customize an approach, most often on a strategic level, often resell tests created by others Solution providers- Use technology infastructure to customize solutions for clients, often embed assessment within a suite of other products Talent management firms- Offer assessment as an add on to other post-hire services. Non-traditional vendors- usually based on some proprietary or revolutionary idea, often deviate from acceptable range
    59. 59. Differentiating vendorsKey parameters of differentiation  Content  Implementation/Configuration model  Technology  Contextual products/services  Scalability  Business partnership  Cost
    60. 60. Where to find vendor info Internet Experience Vendor Expert Advisors List Trade shows, publication s
    61. 61. Initial data collection Review website Make contact Ask pre-defined qualifying questions based on key parameters Ask about examples of results for your industry or for specific jobs of interest Request technical manuals (not just product brochures)
    62. 62. Narrow the list Review data collected Structured RFI (formal or informal) Discuss as a group Begin to track pros and cons Result of process should be list of 3-5 vendors
    63. 63. Red flags Exaggerated claims of impact (90% accurate) Failure to listen well, provide correct info Lack of product that directly meets your needs Inability to tell what they do after discussion with representative and review of materials Poor customer service level No technical documentation No I/O psychologists Lack of expertise in your industry Trash talking
    64. 64. Step 3: Decision making Formalized data collection (RFP, RFI) Example assessments Demos References Team evaluation
    65. 65. Decision makingGoal = Thoroughly evaluate finalist vendors and understand how they differUse this opportunity to: ▪ Eliminate any surprises ▪ Choose the partner who will best deliver the results dictated in your charter ▪ Understand key tradeoffs/compromises related to each vendor ▪ Fully understand a vendor’s capabilities and approach ▪ Learn how they have served others similar to you
    66. 66. RFP & Demos Does not have to be formal to be effective Collect data on key parameters and charter Provides a structured way for all team members to evaluate vendors on key parameters Provides data on which to make decisions Allows you a chance to see how well vendors listen
    67. 67. Key actionHave all team members, including end users take the assessments
    68. 68. Final decision Review pricing and service level Take reference checking seriously Put vendors on the spot! Use scoring from team evaluation process Involve team members Begin shifting gears to contracting and implementation mode
    69. 69. TakeawaysThere is no free lunch! No matter what the need, it is worth it to do your homework Many vendors can help you but there is ROI in optimizing to your unique needs No need to do it all at once, choose one area and gain proof of concept Make the evaluation process your own- adjust the level of rigor based on your needs Assessment is complex – it pays to include an expert
    70. 70. Resources Society for Industrial Organizational Psychology (SIOP) I/O user groups on LinkedIn SHRM Rocket-Hire- Buyer’s Guide
    71. 71.  Industrial/Organizational Psychologist Selection systems specialist Vendor neutral Strong statistical training Futurist and technologist Practical guy who loves ROI!
    72. 72. Questions ??? Look for slides and archive on ERE and