Measurement equivalence of
    Business-focused Inventory of
            Personality :
    A comparison of European langua...
To set the scene

• An international organisation wishes to use a
  personality test to select managers globally for
  exp...
Measurement equivalence
• The relative comparability of the wording, scaling,
  and scoring of constructs across groups

 ...
Levels of Measurement Equivalence
           • Constructs have the same basic factor structure
             across groups
...
Partial Invariance

• Full invariance: ideal but often impossible

• Partial invariance: some, but not all, of the item ar...
Business-focused Inventory of Personality (BIP)

• A work based personality test developed in Germany
  and was adapted to...
The analyses

• Testing measurement equivalence
   – Multi-group exploratory structural equation modelling (ESEM)

• Ident...
Testing measurement equivalence

• Data: German, English, Denmark, Portugal
   – Equal structure: structural equivalence
 ...
Results
• Fit indices:
  – Comparative Fit Index (CFI): >0.95 good fit
  – The root mean square error of approximation (RM...
Full invariance items
                     Scales     Full invariant items
       Achievement Motivation            5
    ...
Differential item functioning
• People from different groups with the same
  underlying ability/trait level have a differe...
MIMIC approach to DIF detecting

                       Country
                                  Item

                  ...
Example
 • BIP Openness to Contact scale
 • Portuguese vs. German.


                         Assume no DIF   DIF effect m...
Measurement equivalence involving emic items

• Etic vs. Emic
   — using the “same” items vs. using culturally specific it...
Missing data technique
• Introducing “imaginary” observed items
               Country A                      Country B

 ...
Example
• BIP Flexibility scale: Spanish vs. German
   – 12 common items
   – 2 items unique to German version
   – 1 item...
Summary

• Measurement equivalence
  – All BIP scales demonstrated structural invariance
  – Most scales showed metric inv...
Implications

• Common items make it possible to equate
  scores across versions in the presence of DIF

• Comparing instr...
Thank you


tao.li@hogrefe.co.uk
Upcoming SlideShare
Loading in …5
×

ITC Measurement equivalence

711 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
711
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ITC Measurement equivalence

  1. 1. Measurement equivalence of Business-focused Inventory of Personality : A comparison of European language versions Tao Li Hogrefe Ltd. UK The 7th Conference of the International Test Commission Hong Kong July 2010
  2. 2. To set the scene • An international organisation wishes to use a personality test to select managers globally for expatriate assignments – Does the test measure the same traits for the candidates? – Are the scores comparable across countries?
  3. 3. Measurement equivalence • The relative comparability of the wording, scaling, and scoring of constructs across groups – A prerequisite for valid group comparison – Implicitly assumed but RARELY examined
  4. 4. Levels of Measurement Equivalence • Constructs have the same basic factor structure across groups Structural • The constructs have similar meaning • The strength of the relationships between items and constructs being measured are equivalent Metric • The constructs have the same meaning • Measure the constructs on the same scale • The groups use the response scale in a similar way Scalar • Complete comparability of scores
  5. 5. Partial Invariance • Full invariance: ideal but often impossible • Partial invariance: some, but not all, of the item are equivalent across groups
  6. 6. Business-focused Inventory of Personality (BIP) • A work based personality test developed in Germany and was adapted to all major European languages – R. Hossiep & M. Paschen, 1998, 2003 © by Hogrefe • An combination of etic-emic approach to adaptation – Etic : e.g. English, Portugal, Dutch, Denmark – Emic: e.g. French, Spanish
  7. 7. The analyses • Testing measurement equivalence – Multi-group exploratory structural equation modelling (ESEM) • Identifying differential item functioning (DIF) – Multiple indicators-multiple causes approach (MIMIC) • Testing measurement equivalence involving emit items – Missing data technique
  8. 8. Testing measurement equivalence • Data: German, English, Denmark, Portugal – Equal structure: structural equivalence – Equal loading: metric equivalence – Equal intercept: scalar equivalence
  9. 9. Results • Fit indices: – Comparative Fit Index (CFI): >0.95 good fit – The root mean square error of approximation (RMSEA): <0.08 good fit • Structural equivalence: all scales – min CFA=0.956; max RMSEA=0.070 • Metric equivalence: 11/14 scales – min CFA=0.950; max RMSEA=0.060 • Scalar equivalence: none • Partial equivalence: all scales – min CFA=0.950; max RMSEA=0.055
  10. 10. Full invariance items Scales Full invariant items Achievement Motivation 5 Power Motivation 3 Leadership Motivation 3 Conscientiousness 3 Flexibility 3 Action Orientation 4 Social Sensitivity 4 Openness to Contact 4 Sociability 3 Team Orientation 2 Assertiveness 3 Emotional Stability 2 Working under Pressure 3 Self confidence 4
  11. 11. Differential item functioning • People from different groups with the same underlying ability/trait level have a different probability of endorsing an item • MIMIC approach to DIF detecting – Modelling DIF and latent mean difference simultaneously
  12. 12. MIMIC approach to DIF detecting Country Item Item Construct Item Item Item
  13. 13. Example • BIP Openness to Contact scale • Portuguese vs. German. Assume no DIF DIF effect modelled Latent mean difference 0.20 0.52 Effect size Small Medium
  14. 14. Measurement equivalence involving emic items • Etic vs. Emic — using the “same” items vs. using culturally specific items • How to compare combined etic-emic instruments?
  15. 15. Missing data technique • Introducing “imaginary” observed items Country A Country B Common Common item item Common Common item item Common Common item item Construct Construct Emic item A Emic item B Imaginary Imaginary Emic item B Emic item A
  16. 16. Example • BIP Flexibility scale: Spanish vs. German – 12 common items – 2 items unique to German version – 1 items unique to Spanish version Model fit CFI: 0.978; RMSEA: 0.046 Latent mean difference 0.10
  17. 17. Summary • Measurement equivalence – All BIP scales demonstrated structural invariance – Most scales showed metric invariance – No scales presented scalar invariance – Full invariant items were identified for each scale
  18. 18. Implications • Common items make it possible to equate scores across versions in the presence of DIF • Comparing instruments involving emic items is possible and necessary
  19. 19. Thank you tao.li@hogrefe.co.uk

×