presentation done as a part of the subject Performance management system studying as to what does validity means in performance appraisal with lot of data

1. Validity in Performance Appraisal
Presented by
MOHAMMED SAQIB

2. Meaning
• Validity involves the interpretation of
observed scores as representative of some
external property (Kane, 1982)
• Reliability deals with the consistency of the
observed scores

3. • The reliability of the measurement procedure
is a relative term and varies widely
• The validity of a measurement is a matter of
degree that depends on the design of the
measurement procedure and in the
interpretation of the attribute measured
• Generally the evidence of the validity is less
when compared to the reliability in
performance appraisal

4. validity
• Validity is a technical term that has to do with
the accuracy and relevance of measurements.
• It is the most critical issue in the appraisal
process and the specialist has to be very
careful
• practitioners must withstand legal challenges
to their performance appraisal tools and
procedures

5. Validation strategies and how they
apply to the examination of
performance appraisal

6. Cronbach’s approach
“validation is an inquiry into the soundness of an
interpretation”
• Validation is taken as the hypothesis which
should be tested and the results should be
supported with the findings
• the researcher is continually recognizing rival
hypotheses and testing them—the result is a
greater understanding of the inferences that can
be made about the characteristics of the
individuals who take a test or who are measured
on a performance appraisal scale.

7. Success story
Testing to select aircraft crew members during
World War II
From Article titled "Validity for What," by
Jenkins (1946)

8. Three major validation strategies given
by Jenkins
1. Construct validation
2. Content validation
3. Criterion-related
Traditional approach: these are separate
Modern approach: Validity is the integration of the
above
(Landy, 1986; Wainer and Braun, 1988; Cronbach,
1990)

9. Construct validation: Is the continuous process and Many pieces of
evidence required
Example: to find the attributes which clearly indicate why two trainees
who have joined on same day have the different ratings
Content validation: Explores the match between the content of the
measure and the content of the job
Example: A test of typing speed and accuracy for a clerk/typist job
Criterion-related: statistically demonstrates the relationships between
people's scores on a measurement instrument and their scores on
the performance of interest
Example: scores on an employment test and supervisor ratings of on-
the-job performance

10. Traditional Construct validation
Types of evidences
• Convergent evidence
the measure in question is related to the
other measures of the same construct
• Discriminant validity
– given measure of a construct has a weak
relationship with measures of other constructs
– Stronger than the convergent because this
challenges the rival hypothesis

11. Measurement of construct validity
Campbell and Fiske’s multimethod-multirater method
• 2 or more appraisers
• 2 different rating methods
• Same employee
Results
• Convergent validity is demonstrated by the
agreement among appraisers across rating
methods
• Discriminant validity is demonstrated by the
degree to which the rates are able to distinguish
among the performance dimensions.

12. Content validation
• Based on type of analysis used in developing the appraisal
instrument
• If detailed job analyses or critical incident techniques were used
and behaviourally based scales were developed, it has been
generally assumed that the appraisal instruments have
content validity.
• behaviours placed on the performance dimension scales look like
they are representative of the behaviours involved in performing
the job and they have been judged by the subject matter experts to
be so
• is not particularly useful in judging the managerial performance
because managers do many things that cannot be linked
unambiguously to the accomplishment of specific tasks
• it will be of little value in making the link between job behaviours
and effective performance.

13. Criterion validation
• it is with selection tests used to predict later
performance
• Stuffs like attendance, tardiness, accidents,
measures of output, or other indices are
judged

14. John Hunter's meta-analysis
• Performance measures: Job performance and
supervisor’s ratings
• Study was conducted on the following
1. military enlisted jobs(4) like armor crewman,
armor repairman, cook, and supply specialist
2. civilian jobs(10) like cartographer, customs
inspector, medical laboratory worker, and
firefighter

15. Hunter’s argument
• whether supervisor ratings are determined
entirely by job performance or whether the
ratings are influenced by the employee's job
knowledge
Conclusion
• job knowledge is twice as important as job
performance in the determination of
supervisor ratings

16. Validity and demographic
characteristics
What are demographic characteristics..?
– Race
– Age
– Gender
– Marital status etc.,
Hypothesis
performance ratings are unaffected by the existence of
these systematic sources of bias

17. Kraiger and Ford's Survey
(Race)
• 74 studies were conducted
• Hypothesis was rejected
• Raters gave good ratings to same race
subordinates
• Average ratings to the same race subordinates
were 64% higher than to the different race
subordinates

18. Carson et al and Murphy et al’s Survey
(Gender)
• 24 studies were conducted
• the gender of both the ratee and the rater
accounted for less than 1% of the variance in
ratings

19. McEvoy and Cascio’s survey
(Age)
• 96 studies were conducted
• the age of the ratee accounted for less than
1% of the variance in performance ratings.
• Landy and Farr also suggest that if age effects
exist at all, they are likely to be very small.

20. Training to the appraiser
• Feldman (1986) concluded that rater training has not
been shown to be highly effective in increasing
the validity and accuracy of ratings.
• Murphy and Cleveland (1991) suggest that training is
required when the underlying problem is a lack of
knowledge or understanding. For example, training is
more necessary if the performance appraisal system
requires complicated procedures, calculations, or
rating methods. However, these authors also suggested
that the accuracy of overall or global ratings will not be
influenced by training.

21. Validating a test
Steps
1. Analyze the job
2. Select the test
3. Administer the test
4. Relate the test scores to the actual
performance
5. Cross validate and re validate

22. Is Validity defensible..??
Yes. If it is,
• Based on Job analysis
• Standardized
• Consistent
• validated