Psychometrics Slides


Published on

Psychometrics Slides Breakfast Briefing on 15th February by from Emma Rush

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Psychometrics Slides

  1. 1. Psychometrics 101 Emma RushSession for the Legal Education and Training Group Emma Rush Consulting February 2011
  2. 2. AgendaGain a better understanding of the ‘added value’ apsychometric can bring within a legal environmentGet a clear overview of the six main instruments onthe market and when they can be usedUnderstand the resource issues of psychometrics:the cost, the time required, the trained personnelneededDevelop your own case, if required, for introducingpsychometrics to all or any of the ‘people processes’ Emma Rush Consulting
  3. 3. Starting with meAs a consultant: Hogan™, MBTI Step II® and OPQ32r™ for coaching and career consultingPrevious law firm: 16PF® for Assessment for Development Centre MBTI Steps I & II ® for team workshops FIRO ® for Leadership programmeFSI: 16PF ® mapped on to a competency framework for partner recruitment and partnership track FIRO® for Assertiveness Destiny for secretarial recruitment Emma Rush Consulting
  4. 4. Brief history of psychometrics Began in late C19th with “intelligence” testing Francis Galton, UK: Anthropometrics – taking a mathematical approach to the measure of individual differences – eugenics Charles Spearman, USA 1910 – contributed ‘g’ or the measurement of general intelligence factor And “factor analysis” – the means by which psychometric tests are found to be valid Emma Rush Consulting
  5. 5. Brief history of psychometrics 1917 Stanford-Binet test: first standardised test of intelligence 1949: First publication of the 16PF by Raymond Cattell (Spearman’s student) 1962: MBTI first published – type-based instrument; four preference scales Emma Rush Consulting
  6. 6. Brief history of psychometrics1963 First appearance of the “Big Five” ofpersonality factors1970: Cronbach (Cronbach’s Alpha):emphasises the importance of validity – beingable to predict accurately what a person willdo Emma Rush Consulting
  7. 7. Brief history of psychometrics1984: OPQ32 introduced, published by SHLR&D by Peter Saville and Roger Holdsworth1986: Hogan’s Personality Inventory firstpublished2004 Saville and Holdsworth ousted fromSHL’s board2008 Saville’s Personality Questionnaire /Wave introduced Emma Rush Consulting
  8. 8. Key conceptsReliability: The degree to which aninstrument measures the same way each time itis used under the same conditions with thesame subjects – its “consistency”Validity: The extent to which you actuallyare measuring what you are professing tomeasureType vs. Trait: Type gives you a preference,trait measures how you compare to a widerpopulation Emma Rush Consulting
  9. 9. Where to use psychometrics Used in thousands of organisations at all stages of the employee “life cycle” We’re looking at three: Selection; Development ; Promotion Emma Rush Consulting
  11. 11. Why select with psychometrics?Type of intervention Validity ( r ) where 1.0 is perfect predictionGeneral Mental Ability Tests .51Structured Employment Interviews .51Used together .63Job Knowledge Tests .48Unstructured Employment Interviews .38Assessment Centres .37Reference checks .26Job experience (years) .18Education (years) / Interests .10Graphology .02Chance 0 Emma Rush Consulting
  12. 12. Why select with psychometrics? Emma Rush Consulting
  13. 13. Why select (or train) withpsychometrics? To make the business case you first need to know: What it currently costs (directly and indirectly) to recruit, train and retain someone so you can measure/prove an improvement Exactly what you are looking for – job criteria, competency framework What outcomes you require from training interventions Emma Rush Consulting
  14. 14. In addition testing…Allows for better and clearer discrimination betweencandidates – “beyond the good egg factor”Supports your diversity initiatives: more transparent,fairer and no adverse impactAllows a better fit with your job criteria orcompetency frameworkAdministrative convenience (depending on yourresources)Costs and development time are reasonable Emma Rush Consulting
  15. 15. What do you test?Five key areas to measure How do they get measured? Knowledge Areas on which traditional or unstructured approaches Experience concentrate – psychometrics can still assess and assist Behaviours Areas where online tests and structured approaches can assess Personality these areas and increase validity Capability of recruitment In descending order of ease to Motivation change – motivation hardest Emma Rush Consulting
  16. 16. Intellectual Ability Job-based or reasoning tests – used frequently at trainee level, but not at higher and more expensive level Standard intellectual ability tests: Verbal reasoning Numerical reasoning Thinking style Emma Rush Consulting
  17. 17. PERSONALITY INSTRUMENTS INUSE Emma Rush Consulting
  18. 18. A word of warning to begin! “ seems that users of some questionnaires become attached to the tests that “look right” or appear to be appropriate, which is known as faith validity..” (Saville, 1975) For every argument there is a counter- argument Emma Rush Consulting
  19. 19. The Big , in in test terms OPQ32r® The 16PF5 ® NEO PI-R™ Hogan Personality Inventory and Development Survey™ Saville Wave Professional Styles® Emma Rush Consulting
  20. 20. How valid are they? Figures taken from Saville’s report How valid is your questionnaire?HighvalidityModeratevalidityChance Emma Rush Consulting
  21. 21. A time comparisonInstrument No of Questions Typical Completion timeOPQ32i 416 60 minutesNEO-PI-R 240 40 minutesWave Professional 216 40 minutesStylesHogan 206 30 minutes16PF5 185 30 minutesSaville’s Personality 72 13 minutesQuestionnaire Emma Rush Consulting
  22. 22. Tests to avoidIpsative TestsTests without appropriate normsTests with low reliability estimatesOld tests that have not been re-evaluated inthe last 10 years.Glossy, packaged tests with no psychometricdetails Emma Rush Consulting
  23. 23. And not to be used for selection DISC: “The test suffers from questionable reliability and unknown validity” MBTI: a “type” based instrument so without norm group validity Emma Rush Consulting
  24. 24. Costs Using psychometrics may have a long term benefit but also has a short term cost Per instrument – from £12 to £298 per report Time: an hour’s minimum feedback Online – no additional cost to set up Reputational risk Emma Rush Consulting
  25. 25. Which tests to use?OPQ32r™Pros: an “Occupational” questionnaire so about the“you at work”“r” is the latest version, and has three rather thanfour item choicesPublished by SHL and designed by Professor PeterSavilleCan be mapped onto a competency frameworkMultiple report options Relatively quick to complete Emma Rush Consulting
  26. 26. OPQ32r™ Relationships with Feelings and People Emotion Influence Emotion Sociability Dynamism Empathy Thinking Style Analysis Creativity and Change Structure Emma Rush Consulting
  27. 27. OPQ32r™ Cons: 32 factors – do they add anything extra? They take longer to give feedback on Overlap between factors e.g. relationships with people / feelings and emotion Now quite old (1984), doesn’t cover to same degree motivations, values, influencing style, leadership Is it an ideal fit with professional services – norm groups used? Hard to get an overview of scores in the reports, more interpretative than detailed Emma Rush Consulting
  28. 28. 16PF®Pros: A combination of five global factors andsixteen primary factors – gives some granularity butnot overwhelmingUsed world-wide for recruitment and developmentCan be mapped onto a competency frameworkFactors relate to universal traits rather than forcedinto three pronged modelReports give practitioner option as well asinterpretative reportsRelatively quick to complete Emma Rush Consulting
  29. 29. 16PF® Cons: Meta-perceptions: candidates frequently report “I do this differently in social situations” Measures drive (obliquely) but not specifically motivations Reasoning questions can’t be relied on as measure of intellectual ability, particularly for lawyers Is it an ideal fit for professional services – norm groups used? Emma Rush Consulting
  30. 30. Hogan Personality Inventory/DevelopmentSurvey/Motivations, Preferences and Values Inventory(all™) Pros: Uniquely covers both “bright” and “dark” side of personality Covers what drives individuals, what derails them and what they value in an organisation Particularly suited for senior recruits e.g.. lateral hires, as reports based on strengths and competencies for leaders Emma Rush Consulting
  31. 31. Hogan Personality Inventory/DevelopmentSurvey/Motivations, Preferences and Values Inventory(all™) Cons: quite American – language in reports, language in questions Three separate questionnaires to complete Relevance to professional services? Relevance to more junior roles in professional services? Supplied in UK by smaller distributors e.g. Mentis; Psychological Consultancy Emma Rush Consulting
  32. 32. NEOThe NEO PI-R was developed by Costaand McCraeFive domains measured: Neuroticism,Extraversion, Openness to Experience,Agreeableness and Conscientiousness. Emma Rush Consulting
  33. 33. NEOEach five domains is divided into sixfacets: thirty areas of personality whichpredict employees’ work behaviourincluding leadership style, team style,decision-making and stress management.240-item questionnaire either online or bypaper and pencilThe current UK Edition was published in2004. Emma Rush Consulting
  34. 34. Saville Consulting’s Wave ®Professional Styles edition To quote Peter Saville: “Saville Consulting Wave® Professional Styles measures motives, talents, preferred culture and competency potential in one dynamic online questionnaire” A wealth of detail: 4 Clusters; 12 Sections; 36 Dimensions; 108 Facets More reliable Quick to complete Normed for professionals Emma Rush Consulting
  36. 36. Ongoing development: workshopsand coaching Specific learning e.g. leadership, assertiveness Coaching Team workshops Emma Rush Consulting
  37. 37. Why train with psychometrics? The sheer cost of training Have to do something 200 times to be completely proficient in a new skill - accelerates the process of self-awareness Assessing motivation means you can see in advance: How this person likes to work What motivates this person to work What types of goals this person likes Emma Rush Consulting
  38. 38. What type of intervention?How you like to learn e.g. Honey andMumford’s Learning Styles QuestionnaireWhat motivates you – how do we reinforcelearning e.g. Hogan MPVIWhy you need to learn e.g. FIRO-B withleadershipHow you operate in a team e.g. MBTI Step IIWhat the real issues are in coaching e.g.Saville Consulting Wave® Professional Styles Emma Rush Consulting
  39. 39. Team Interventions Three instances you may instigate a team workshops: Forming a team Dealing with team performance Team building Emma Rush Consulting
  40. 40. FIRO-BA unique way of looking at how a teamoperatesThree “scales”, no typology, scored from 1 to9Inclusion; Control; Affection (or Openness)An “Expressed” score: what you show toothersA “Wanted” score: what you want fromothers but may not show Emma Rush Consulting
  41. 41. Team Workshops MBTI Steps I and II Uniquely lends itself to group feedback with a reported type followed by self-assessed type and ending with best fit Step II adds factors to the four dimensions giving more granularity and better comparison between team members Can be a powerful aide to raising self- awareness Emma Rush Consulting
  42. 42. However… Can feel over-simple even at Step II MBTI Step I gives only four dimensions for comparison – does this adequately explain how a team interacts or personality works? Slip between “reported type” and “best fit” can be as much as 25% e.g.. only 75% of people confirm their MBTI results Lacks real scientific validity Emma Rush Consulting
  43. 43. BelbinDr Meredith Belbin; dates from the 1970s Belbin’s Nine Team Roles Plant Co-ordinator Monitor-Evaluator Resource- Investigator Implementer Completer-finisher Team worker Shaper Specialist Emma Rush Consulting
  45. 45. A form of selection The same instrument should be used for promotion purposes as for selection for reasons of: Consistency Fairness Transparency Proper metrics of progress (from selection to now) Emma Rush Consulting
  46. 46. How do you make the bestdecision on promotion? The more structured methods you use the better validity you gain Assessment centres alone are less valid than when combined with e.g. Structured interviews Psychometrics Emma Rush Consulting
  47. 47. Questions? Emma Rush Consulting
  48. 48. SuppliersOPP: Feedback: Consulting: Emma Rush Consulting
  49. 49. Organisations The Psychometrics Forum Emma Rush Consulting
  50. 50. Training I recommend working out which test(s) you want to use, and then qualifying with that test publisher For example OPP’s Level B course qualifying you in both MBTI® and 16PF® Or Saville Consulting qualifying you in Wave™ Emma Rush Consulting