VALIDITY
Mr. Maneesh Sharma
Assistant Professor
AIIMS,Rishikesh
Introduction
• Important criteria for evaluating a quantitative instrument
• Instrument can not validly measure a attribute if it is inconsistent & inaccurate
• Validity & reliability are not independent qualities of an instrument
Note ; High reliability of instrument provide no evidence of its validity
Definition
Defined as the degree to which an instrument is measuring the construct it purports
the measure.
Polit Beck 2017
Validity refers to degree to which an instrument measures what it is supposed to be
measuring
Polit & Hungler 2012
Types of Validity1. Face validity
2. Content Validity
3. Criterion Validity
- Concurrent Validity
- Predictive validity
4. Construct validity
- Convergent validity
- Divergent Validity
Face validity
• Face validity refers to whether the instrument looks as though it is measuring the appropriate construct.
• Overall look of instrument regarding its appropriateness to measure particular attribute or phenomena
• This is the least sophisticated measure of validity. Face validity is simply whether the test appears (at face
value) to measure what it claims to.
• It is not considered primary evidence for an instrument's validity
• It is helpful for measure if other types of validity also been demonstrated.
(Face value or outlook of instrument )
E.g- Likert scale is designed to measure the attitude of nurses towards HIV patients, research may judge the face value of instrument
by its appurtenance that it looks good or not but it does not provide any guarantee about its appropriateness & completeness
Content Validity
• Content validity concerns the degree to which an instrument has an appropriate sample of
items for the construct being observed.
or
• It is concerned with scope of coverage of content area to be measured.
• It is applied in test of knowledge measurement
• It is a case of expert judgment about content area included in research instrument to
measure a phenomena.
• Judgement of the content viability may be subjective & is based on previous researchers &
expert opinion about the adequacy, appropriateness & completeness of content of instrument.
Cont..
• Measures both
• Affective measures - feeling, emotions & psychological traits
• Cognitive measures – how representatives are the questions …?
Example 1- if researcher want to test student knowledge about nursing theories, the test would not be
content valid if it omitted questions about Orem’s self care theory.
Example 2- Researcher developed a self report scale to measure distress from fatigue in cancer patients .
Items for scale were drawn from 23 in depth interviews with patients experiencing cancer related fatigue.
(Qualitative data for content validity )
Cont …
• An Instrument content validity is necessarily based on judgment
• Panel of experts to evaluate and document the content validity of instrument.
• Minimum of three experts & may be more – depends of complexibility
• Experts are asked to evaluate individual items as well as the entire instrument
- whether individual items are relevant & appropriate in terms of construct
- whether the items adequately measure all dimensions of construct
• With regards to item relevance some researcher compute interrater agreement indexes & formal
content validity index (CVI)
Cont…
• One procedure is expert rate item on 4 point scale
1 – not relevant
2- somewhat relevant
3- quite relevant
4-highly relevant
Then for each item, item CVI (ICVI) computed
• No of experts giving rating divided by no of experts.
• CVI for total instrument is the proportion of items rated as either 3 or 4.
• A CVI score of 0.80 or better indicates good content validity.
Example – A researcher developed a scale to tap nurses acknowledgment of using intuition in clinical decision making. In first
phase of study scale item were generated from published literature & CVI of 0.96 was computed on responses from panel of five
experts.
Component of content validity
• Relevance – relevance of individual items & overall set of items
each items is relevant to construct or specific dimension of
construct …???
whether items have relevance for target population..?
• Comprehensiveness –
see for any notable omission …?
• Balance – content valid instrument represent domains of construct in a balanced
manner.
Criterion Validity
• It involves deterring the relationship between an instrument & external criteria.
• The instrument is said to be valid if its score correlate highly with scores on criteria.
• It is the extent to which the scores on an instrument are a good reflection of a “gold
standard”- that is a criterion considered an ideal measure of construct.
Cont..
• One requirement of criterion validity is the availability of valid & reliable criteria,
with which instrument can be compared.
• Once criterion is selected its easy to asses criterian validity.
• A correlation coefficient is computed between scores on the the instrument & the
criteria.
• Magnitude of correlation coefficient will estimate how valid instrument is.
• Why there is need of focal measure, when we have god standard ….?
Cont..
• Expense – e.g self report measure of physical function is less costly than a battery
of physical performance test
• Efficiency – e.g if 2 min. walk test can yield comparable information to 6 min .walk
test then 2 min. walk test sometimes might be preffered
• Risk & comfort – puts people at risk or is invasive, a substitute is desired to lower
risk or pain
• Criterion unavailable
• Prediction
Types
1. Concurrent Validity
2. Predictive validity
Concurrent Validity
This is the degree to which a test corresponds to an external criterion that is known
concurrently (i.e. occurring at the same time). If the new test is validated by a comparison
with a currently existing criterion, we have concurrent validity.
E.g - A Psychological test to differentiate between patient in mental hospital who can
& can not be released could be correlated with current behavioural ratings of health
care personnels.
Predictive Validity
• It refers to the adequacy of an instrument in differentiating between people’s
performance on some future criterion.
• This is the degree to which a test accurately predicts a criterion that will occur in the future.
Eg. A prediction may be made on the basis of a new intelligence test, that high scorers
at age 12 will be more likely to obtain university degrees several years later. If the
prediction is born out then the test has predictive validity.
Construct Validity
• Construct validity as the degree to which evidence about a measures' scores in
relation to other scores supports the inference that the construct has been
appropriate represented.
e.g- Researcher may have designed the instrument to measure the concept of pain in
imputed patients. The pain pattern is due to anxiety too; hence result is misleading.
So the questions comes…………..?
Cont…
• What is the instrument really measuring ………….????
• Does it adequately measuring the abstract concept of interest …………..????
• This type of validity refers to the extent to which a test captures a specific theoretical construct or
trait.
• If you can demonstrate that your measure matches with what theories & other studies say about the
variable.
Types of construct Validity
1. Convergence validity
2. Divergence validity
Summary
Thank you

Validity

  • 1.
    VALIDITY Mr. Maneesh Sharma AssistantProfessor AIIMS,Rishikesh
  • 2.
    Introduction • Important criteriafor evaluating a quantitative instrument • Instrument can not validly measure a attribute if it is inconsistent & inaccurate • Validity & reliability are not independent qualities of an instrument Note ; High reliability of instrument provide no evidence of its validity
  • 3.
    Definition Defined as thedegree to which an instrument is measuring the construct it purports the measure. Polit Beck 2017 Validity refers to degree to which an instrument measures what it is supposed to be measuring Polit & Hungler 2012
  • 4.
    Types of Validity1.Face validity 2. Content Validity 3. Criterion Validity - Concurrent Validity - Predictive validity 4. Construct validity - Convergent validity - Divergent Validity
  • 5.
    Face validity • Facevalidity refers to whether the instrument looks as though it is measuring the appropriate construct. • Overall look of instrument regarding its appropriateness to measure particular attribute or phenomena • This is the least sophisticated measure of validity. Face validity is simply whether the test appears (at face value) to measure what it claims to. • It is not considered primary evidence for an instrument's validity • It is helpful for measure if other types of validity also been demonstrated. (Face value or outlook of instrument ) E.g- Likert scale is designed to measure the attitude of nurses towards HIV patients, research may judge the face value of instrument by its appurtenance that it looks good or not but it does not provide any guarantee about its appropriateness & completeness
  • 6.
    Content Validity • Contentvalidity concerns the degree to which an instrument has an appropriate sample of items for the construct being observed. or • It is concerned with scope of coverage of content area to be measured. • It is applied in test of knowledge measurement • It is a case of expert judgment about content area included in research instrument to measure a phenomena. • Judgement of the content viability may be subjective & is based on previous researchers & expert opinion about the adequacy, appropriateness & completeness of content of instrument.
  • 7.
    Cont.. • Measures both •Affective measures - feeling, emotions & psychological traits • Cognitive measures – how representatives are the questions …? Example 1- if researcher want to test student knowledge about nursing theories, the test would not be content valid if it omitted questions about Orem’s self care theory. Example 2- Researcher developed a self report scale to measure distress from fatigue in cancer patients . Items for scale were drawn from 23 in depth interviews with patients experiencing cancer related fatigue. (Qualitative data for content validity )
  • 8.
    Cont … • AnInstrument content validity is necessarily based on judgment • Panel of experts to evaluate and document the content validity of instrument. • Minimum of three experts & may be more – depends of complexibility • Experts are asked to evaluate individual items as well as the entire instrument - whether individual items are relevant & appropriate in terms of construct - whether the items adequately measure all dimensions of construct • With regards to item relevance some researcher compute interrater agreement indexes & formal content validity index (CVI)
  • 9.
    Cont… • One procedureis expert rate item on 4 point scale 1 – not relevant 2- somewhat relevant 3- quite relevant 4-highly relevant Then for each item, item CVI (ICVI) computed • No of experts giving rating divided by no of experts. • CVI for total instrument is the proportion of items rated as either 3 or 4. • A CVI score of 0.80 or better indicates good content validity. Example – A researcher developed a scale to tap nurses acknowledgment of using intuition in clinical decision making. In first phase of study scale item were generated from published literature & CVI of 0.96 was computed on responses from panel of five experts.
  • 10.
    Component of contentvalidity • Relevance – relevance of individual items & overall set of items each items is relevant to construct or specific dimension of construct …??? whether items have relevance for target population..? • Comprehensiveness – see for any notable omission …? • Balance – content valid instrument represent domains of construct in a balanced manner.
  • 11.
    Criterion Validity • Itinvolves deterring the relationship between an instrument & external criteria. • The instrument is said to be valid if its score correlate highly with scores on criteria. • It is the extent to which the scores on an instrument are a good reflection of a “gold standard”- that is a criterion considered an ideal measure of construct.
  • 12.
    Cont.. • One requirementof criterion validity is the availability of valid & reliable criteria, with which instrument can be compared. • Once criterion is selected its easy to asses criterian validity. • A correlation coefficient is computed between scores on the the instrument & the criteria. • Magnitude of correlation coefficient will estimate how valid instrument is. • Why there is need of focal measure, when we have god standard ….?
  • 13.
    Cont.. • Expense –e.g self report measure of physical function is less costly than a battery of physical performance test • Efficiency – e.g if 2 min. walk test can yield comparable information to 6 min .walk test then 2 min. walk test sometimes might be preffered • Risk & comfort – puts people at risk or is invasive, a substitute is desired to lower risk or pain • Criterion unavailable • Prediction
  • 14.
  • 15.
    Concurrent Validity This isthe degree to which a test corresponds to an external criterion that is known concurrently (i.e. occurring at the same time). If the new test is validated by a comparison with a currently existing criterion, we have concurrent validity. E.g - A Psychological test to differentiate between patient in mental hospital who can & can not be released could be correlated with current behavioural ratings of health care personnels.
  • 16.
    Predictive Validity • Itrefers to the adequacy of an instrument in differentiating between people’s performance on some future criterion. • This is the degree to which a test accurately predicts a criterion that will occur in the future. Eg. A prediction may be made on the basis of a new intelligence test, that high scorers at age 12 will be more likely to obtain university degrees several years later. If the prediction is born out then the test has predictive validity.
  • 17.
    Construct Validity • Constructvalidity as the degree to which evidence about a measures' scores in relation to other scores supports the inference that the construct has been appropriate represented. e.g- Researcher may have designed the instrument to measure the concept of pain in imputed patients. The pain pattern is due to anxiety too; hence result is misleading. So the questions comes…………..?
  • 18.
    Cont… • What isthe instrument really measuring ………….???? • Does it adequately measuring the abstract concept of interest …………..???? • This type of validity refers to the extent to which a test captures a specific theoretical construct or trait. • If you can demonstrate that your measure matches with what theories & other studies say about the variable.
  • 19.
    Types of constructValidity 1. Convergence validity 2. Divergence validity
  • 20.
  • 21.