SlideShare a Scribd company logo
VALIDITY
AND
RELIABILITY OF
QUESTIONNAIRES
Dr. R. VENKITACHALAM
CONTENTS
 Introduction
 Steps in questionnaire designing
 Validity
 Concept of validity
 Types of validity
 Steps in questionnaire validation
 Reliability
 Types and measurement of reliability
 Conclusion
 References
INTRODUCTION
 Questionnaire: Important method of data collection used
extensively
 Advantages of questionnaire
 Less expensive
 Offers greater anonymity
 Disadvantages
 Application is limited
 Response rate is low
 Opportunities to clarify issues is lacking
 Ideal requisites of a questionnaire:
 Should be clear and easy to understand
 Layout is easy to read and pleasant to eye
 Sequence of questions easy to follow
 Should be developed in an interactive style
 Sensitive questions must be worded exactly
 NOTE: The terminologies research instrument, measuring
instrument,scale and test in various parts of the seminar
represent questionnaire in this context . . . And item represents
each question in a questionnaire
Steps in questionnaire designing
Validity
The concept of validity
 Validity is the ability of an instrument to measure what it is intended to
measure.
 Degree to which the researcher has measured what he has set out to
measure (Smith, 1991)
 Are we measuring what we think we are measuring? (Kerlinger, 1973)
 Extent to which an empirical measure adequately reflects the real
meaning of the concept under consideration (Babbie, 1989)
Why validity ?
 Validity is done mainly to answer the following questions:
 Is the research investigation providing answers to the research
questions for which it was undertaken?
 If so, is it providing these answers using appropriate methods and
procedures?
Questions to ponder
Investigator
Readers of report
Experts in the field
Logic
Statistical tests
Logical thinking
 Justification of each question in relation to objective of study
 Easy if questions relate to tangible matters
 Difficult in situations where we are measuring attitude,
effectiveness of a program, satisfaction etc
 Everybody’s logic doesn’t match . . No statistical backing
Statistical procedures
 By calculating coefficient of correlations between
questions and outcome variables
Types of validity
Validity
Content
validity
Face validity
Criterion
related
Concurrent Predictive
Construct
validity
CONTENT VALIDITY
 Uses logical reasoning and hence easy to apply
 Extent to which a measuring instrument covers a
representative sample of the domain of the aspects measured
 Whether items and questions cover the full range of the
issues or problem being measured
FACE VALIDITY
 The extent to which a measuring instrument appears valid
on its surface
 Each question or item on the research instrument must have a
logical link with the objective
Face validity is not content validity. Why?
 Face validity
 Simply addresses whether a measuring instrument looks
valid
 Not a validity in technical sense because it does not refer
to what is actually being measured rather what it appears
to measure
 It has more to do with rapport and public relations than
with actual validity
Other aspects of content validity
 Coverage of issue should be balanced
 Each aspect should have similar and adequate representation
in questions
Problems associated with content validity
 Based on subjective logic; no definitive conclusion can be
drawn or consensus reached
 Extent to which questions reflect the objectives of the study
may differ. If wordings changed or question substituted,
magnitude of link changes
CRITERION VALIDITY
 The extent to which a measuring instrument accurately
predicts behaviour or ability in a given area.
 The measuring instrument is called ‘criteria’
 It is of two types:
 Predictive validity
 Concurrent validity
Predictive validity
 If the test is used to predict future performance
 Eg: Entrance exam . . . . Performance of these tests correlates
with later performance in professional college
 Eg: Written driving test
 Eg: measurement of sugar exposure for caries development
Concurrent validity
 If the test is used to estimate present performance or person’s
ability at the present time not attempting to predict future
outcomes
 Professional college exam
 Eg: driving test, pilot test
 Eg: measurement of DMFT for caries experience
Problems in criterion validity
 Cannot be used in all circumstances
 Esp in social sciences where some conditions do not have a
relevant criteria
 Eg: for measuring self-esteem, no criteria can be applied
CONSTRUCT VALIDITY
 Most important type of validity
 Assesses the extent to which a measuring instrument
accurately measures a theoretical construct it is designed to
measure
 Measured by correlating performance on the test with
performance on a test for which construct validity has been
determined
 Eg: a new index for measuring caries can be validated by
comparing its values with a standard index (like DMFT)
 Another method is to show that scores of the new test differs
across people with different levels of outcomes being
measured
 Eg: Establishing the validity of a new caries index by
applying it to different stages of dental caries and calculating
its accuracy
Summary of Validity
CONTENT CRITERION CONSTRUCT
CONCURRENT PREDICTIVE
What it
measures
Whether the test
covers a
representative
sample of the
domains to be
measured
The ability of
the test to
estimate present
performance
The ability of the
test to predict
future
performance
The extent to
which the
instrument
measures a
theoretical
construct
How it is
accomplished
Ask experts to
assess the test to
establish that the
items are
representative of
the outcome
Correlate
performance on
the test with a
concurrent
behaviour
Correlate
performance on
the test with a
behaviour in
future
Correlate
performance on
the instrument
with a
performance on
an established
instrument
Steps in
questionnaire
validation
FACE VALIDITY
 Evaluate in terms of:
Readability
Layout
and style
Clarity of wording
Feasibility
CONTENT VALIDITY
Two phases
Researcher: Conceptualization and
domain analysis
Experts: Enhancement of content of
questionnaire (Seven or more
experts)
Specify the full domain of
content that is relevant to
the issue
Sample specific areas form
this domain
Put items/questions in a form
that is testable
How do experts evaluate validity
 Method 1: Average Congruency Percentage (ACP)
[Popham, 1978]
 Experts compute the percentage of questions deemed
to be relevant for them
 Take the average of all experts
 If the value is > 90 . . . Valid
 Eg: 2 experts . . (Expert 1-100%, Expert 2-80%)
 Then ACP = 90%
 Method 2: Content validity index [Martuza 1977]
 Content validity Index for individual items (I-CVI)
 Content Validity Index for the scale (S-CVI)
I-CVI
 Panel of content experts asked to review the relevance of
each question on a 4-point Likert scale (minimum 3
maximum 10 experts)
 1= not relevant
 2= somewhat relevant
 3= relevant
 4= very relevant
 Then for each question, number of experts giving 3 or 4
score is counted (3,4 – relevant; 1,2 – nonrelevant)
 Proportion is calculated
 Eg: If 4/5 experts give score 3 or 4: I-CVI = 0.80
Critics of I-CVI
 Collapses experts multipoint assessment into two categories
(relevant and non-relevant)
 Does not give inference on comprehensiveness of whole
questionnaire
 Problem of chance agreement. To overcome that, Lynn
proposed
 Five or fewer experts: all must agree (I-CVI = 1.0)
 Six or more: (I-CVI should not be less than 0.78)
S-CVI
 The proportion of items on an instrument that achieved a
rating of 3 or 4 by all the content experts
 Two approaches:
 S-CVI/UA – Universal agreement
 S-CVI/Ave - Average
 Which would be an effective measure here ??
 S-CVI/UA or S-CVI/Ave
 Which to follow?
 Report both the values I-CVI and S-CVI rather than using
CVI as an acronym
 Report the range of I-CVI values
 The best method is S-CVI/UA for stringent validity, but
will be difficult to use if multiple experts are validating. .
In such situations S-CVI/Ave is used
CONSTRUCT VALIDITY
 Method: Factor analysis
 To examine empirically the interrelationship among items and to
identify clusters of items that share sufficient variation to justify
their existence as a factor or construct to be measured by the
instrument
 Various items are gathered into common factors
 Common factors are synthesized into fewer factors and then
relation between each item and factor is measured
 Unrelated items are eliminated
Reliability
RELIABILITY
 Definition: It is the ability of an instrument to create
reproducible results
 Each time it is used, similar scores should be obtained
 A questionnaire is said to be reliable if we get same/similar
answers repeatedly
 Though it cannot be calculated exactly, it can be measured
by estimating correlation coefficients
Reliability measured in aspects of:
• Done to ensure that same results are obtained
when used consecutively for two or more times
• Test-retest method is used
STABILITY
• To ensure all subparts of a instrument measure
the same characteristic (Homogeneity)
• Split-half method
INTERNAL
CONSISTENCY
• Used when two observers study a single
phenomenon simulataneously
• Inter-rater reliability
EQUIVALENCE
Test-Retest reliability (for stability)
 Test administered twice to the same participant at different
times
 Used for things that are stable over time
 Easy and straight-forward approach
 Useful for questionnaires, checklist, rating scales etc
 Disadvantages
 Practice effect (mainly for tests)
 Too short intervals in between (effect of memory)
 Some traits may change with time
Statistical calculation
 Administration of instrument to a sample on two
different occasions
 Scores compared and calculated by using
correlation coefficient formula (pearson)
Correlation coefficient
 Measures the degree of relationship between two sets of
scores
 Can range from -1 to +1
 0 indicates absence of any relationships
Correlation coefficient Strength of relationship
+/- 0.7 to 1.0 Strong
+/- 0.3 to 0.69 Moderate
+/- 0.0 to 0.29 None to weak
Split halves reliability (homogenity)
 Split the contents of the questionnaire into two equivalent
halves; either odd/even number or first/second half
 Correlate scores of one half with scores of the other
 Formula: r = Σ (x-x’)(y-y’)
√ Σ(x-x’)2 (y-y’)2
 But this r is only for the half, so to check reliability of
entire test, use the formula
 R’ = 2r/1+r
 (r = coefficient of split half, R’ = coefficient of entire
test)
 Cronbach’s alpha:
 Another method of calculation using the formula:
R = k/k-1 (1-Σσ1
2/σy2)
k = total number of items in list
σ1 = variance of individual items
σy2 = variance of total test scores
Inter-rater reliability (Equivalence)
 Used when a single event is measured simultaneously and
independently by two or more trained observers
R = Number of agreements
Number of agreements + Number of disagreements
Summary of Reliability
TEST RETEST SPLIT HALF INTERRATER
What it
measures
Stability over
time
Equivalency of items Agreement between
raters
How it is
accomplished
Administer the
same test to the
same people at
two different
times
Correlate
performance for a
group of people on
two equivalent
halves of same test
Have multiple
researchers measure
same instrument and
determine percentage
of agreement between
them
Conclusion
 Validated questionnaire
 It is one which has undergone a validation procedure to
show that it accurately measures what it aims to,
regardless of who responds, when they respond, and to
whom they respond or when self-administered and whose
reliability has also been examined thereby:
 Reducing bias and ambiguities
 Better quality of data and credible information
In a nutshell . . . .
A questionnaire can be reliable but invalid . . .
But a valid questionnaire is always reliable . . .
Acknowledgements
 Dr. Joe Joseph
 Dr. Chandrashekar
References
 Linda Del Greco, Walop W, Richard H McCarthy. Questionnaire
development: 2. Validity and Reliability. CMAJ. 1987;136:699–700.
 Sushil S, Verma N. Questionnaire validation made easy. Eur J Sci Res.
2010;46(2):172–8.
 Polit DF, Cheryl Tatano Beck. The Content Validity Index: Are You Sure
You Know What’s Being Reported? Critique and Recommendations. Res
Nurs Health. 2006;29:489–97.
 Reliability and Validity Module 6. Cengage Learning; 2010.
 Rama B Radhakrishna. Tips for Developing and Testing
Questionnaires/Instruments. J Ext. 2007;35(1):710–4.
 06Article04.pdf [Internet]. [cited 2015 Apr 7]. Available from:
http://www.uk.sagepub.com/salkind2study/articles/06Article04.pdf
 pta_6871_6791004_64131.pdf [Internet]. [cited 2015 Apr 7]. Available
from:
http://cfd.ntunhs.edu.tw/ezfiles/6/1006/attach/33/pta_6871_6791004_6
4131.pdf
 Questionnaire designing and validation [Internet]. [cited 2015 Apr 7].
Available from:
http://www.jpma.org.pk/full_article_text.php?article_id=3414
 Suresh K Sharma. Nursing Research and Statistics. 1st ed. New Delhi:
Elsevier Saunders;
 Edward G, Richard Zeller. Reliability and Validity Assessment. New Delhi:
SAGE publication; 1979.
 Ranjit Kumar. Research Methodology - A step by step guide for beginners.
3rd ed. New Delhi: SAGE publication; 2012.
 Articles from Dr. Joe
Validity and reliability of questionnaires

More Related Content

What's hot

Descriptive Statistics
Descriptive StatisticsDescriptive Statistics
Descriptive StatisticsCIToolkit
 
Thematic Analysis for Qualitative Research.pptx
Thematic Analysis for Qualitative Research.pptxThematic Analysis for Qualitative Research.pptx
Thematic Analysis for Qualitative Research.pptxPavaniPrabhaTanaji
 
Introduction to Regression Analysis
Introduction to Regression AnalysisIntroduction to Regression Analysis
Introduction to Regression AnalysisSibashis Chakraborty
 
Validity & reliability seminar
Validity & reliability seminarValidity & reliability seminar
Validity & reliability seminarmrikara185
 
Presentation Validity & Reliability
Presentation Validity & ReliabilityPresentation Validity & Reliability
Presentation Validity & Reliabilitysongoten77
 
Reliability & Validity
Reliability & ValidityReliability & Validity
Reliability & ValidityIkbal Ahmed
 
200 chapter 7 measurement :scaling by uma sekaran
200 chapter 7 measurement :scaling by uma sekaran 200 chapter 7 measurement :scaling by uma sekaran
200 chapter 7 measurement :scaling by uma sekaran Irfan Sheikh
 
What is Reliability and its Types?
What is Reliability and its Types? What is Reliability and its Types?
What is Reliability and its Types? Dr. Amjad Ali Arain
 
Chi Square Worked Example
Chi Square Worked ExampleChi Square Worked Example
Chi Square Worked ExampleJohn Barlow
 
Data Collection-Primary & Secondary
Data Collection-Primary & SecondaryData Collection-Primary & Secondary
Data Collection-Primary & SecondaryPrathamesh Parab
 
Questionnaire and its Types
Questionnaire and its Types Questionnaire and its Types
Questionnaire and its Types Mumbai University
 
Measurement, scaling and sampling
Measurement, scaling and samplingMeasurement, scaling and sampling
Measurement, scaling and samplingRajThakuri
 
8. validity and reliability of research instruments
8. validity and reliability of research instruments8. validity and reliability of research instruments
8. validity and reliability of research instrumentsRazif Shahril
 
Good scale measurement
Good scale measurementGood scale measurement
Good scale measurementsai precious
 
Qualitative Research Method
 Qualitative Research  Method  Qualitative Research  Method
Qualitative Research Method Kunal Modak
 

What's hot (20)

Descriptive Statistics
Descriptive StatisticsDescriptive Statistics
Descriptive Statistics
 
Thematic Analysis for Qualitative Research.pptx
Thematic Analysis for Qualitative Research.pptxThematic Analysis for Qualitative Research.pptx
Thematic Analysis for Qualitative Research.pptx
 
Introduction to Regression Analysis
Introduction to Regression AnalysisIntroduction to Regression Analysis
Introduction to Regression Analysis
 
Validity & reliability seminar
Validity & reliability seminarValidity & reliability seminar
Validity & reliability seminar
 
Presentation Validity & Reliability
Presentation Validity & ReliabilityPresentation Validity & Reliability
Presentation Validity & Reliability
 
Reliability & Validity
Reliability & ValidityReliability & Validity
Reliability & Validity
 
200 chapter 7 measurement :scaling by uma sekaran
200 chapter 7 measurement :scaling by uma sekaran 200 chapter 7 measurement :scaling by uma sekaran
200 chapter 7 measurement :scaling by uma sekaran
 
Concept, Construct and Variable
Concept, Construct and VariableConcept, Construct and Variable
Concept, Construct and Variable
 
What is Reliability and its Types?
What is Reliability and its Types? What is Reliability and its Types?
What is Reliability and its Types?
 
Chi Square Worked Example
Chi Square Worked ExampleChi Square Worked Example
Chi Square Worked Example
 
Reliability
ReliabilityReliability
Reliability
 
Quantitative Data analysis
Quantitative Data analysisQuantitative Data analysis
Quantitative Data analysis
 
Data Collection-Primary & Secondary
Data Collection-Primary & SecondaryData Collection-Primary & Secondary
Data Collection-Primary & Secondary
 
q sort
 q sort q sort
q sort
 
Questionnaire and its Types
Questionnaire and its Types Questionnaire and its Types
Questionnaire and its Types
 
Reliability
ReliabilityReliability
Reliability
 
Measurement, scaling and sampling
Measurement, scaling and samplingMeasurement, scaling and sampling
Measurement, scaling and sampling
 
8. validity and reliability of research instruments
8. validity and reliability of research instruments8. validity and reliability of research instruments
8. validity and reliability of research instruments
 
Good scale measurement
Good scale measurementGood scale measurement
Good scale measurement
 
Qualitative Research Method
 Qualitative Research  Method  Qualitative Research  Method
Qualitative Research Method
 

Similar to Validity and reliability of questionnaires

Validity and reliability in assessment.
Validity and reliability in assessment. Validity and reliability in assessment.
Validity and reliability in assessment. Tarek Tawfik Amin
 
Validity and reliability
Validity and reliabilityValidity and reliability
Validity and reliabilityrandoparis
 
Assessing learning in Instructional Design
Assessing learning in Instructional DesignAssessing learning in Instructional Design
Assessing learning in Instructional Designleesha roberts
 
research-instruments (1).pptx
research-instruments (1).pptxresearch-instruments (1).pptx
research-instruments (1).pptxJCronus
 
JC-16-23June2021-rel-val.pptx
JC-16-23June2021-rel-val.pptxJC-16-23June2021-rel-val.pptx
JC-16-23June2021-rel-val.pptxsaurami
 
Test characteristics
Test characteristicsTest characteristics
Test characteristicsSamcruz5
 
Validity, reliability and feasibility
Validity, reliability and feasibilityValidity, reliability and feasibility
Validity, reliability and feasibilitysilpa $H!lu
 
Evaluation in Education
Evaluation in EducationEvaluation in Education
Evaluation in EducationKusum Gaur
 
Validity and objectivity of tests
Validity and objectivity of testsValidity and objectivity of tests
Validity and objectivity of testsbushra mushtaq
 
Measurement and scaling
Measurement and scalingMeasurement and scaling
Measurement and scalingBalaji P
 
Valiadity and reliability- Language testing
Valiadity and reliability- Language testingValiadity and reliability- Language testing
Valiadity and reliability- Language testingPhuong Tran
 
D8 and d9 personality test development 10 2007-posting
D8 and d9 personality test development 10 2007-postingD8 and d9 personality test development 10 2007-posting
D8 and d9 personality test development 10 2007-postingBlessed Santos
 
Validity, Reliability and Feasibility
Validity, Reliability and FeasibilityValidity, Reliability and Feasibility
Validity, Reliability and FeasibilityJasna3134
 

Similar to Validity and reliability of questionnaires (20)

Week 8 & 9 - Validity and Reliability
Week 8 & 9 - Validity and ReliabilityWeek 8 & 9 - Validity and Reliability
Week 8 & 9 - Validity and Reliability
 
Validity and reliability in assessment.
Validity and reliability in assessment. Validity and reliability in assessment.
Validity and reliability in assessment.
 
Validity and reliability
Validity and reliabilityValidity and reliability
Validity and reliability
 
7.1 assessment and the cefr (1)
7.1 assessment and the cefr (1)7.1 assessment and the cefr (1)
7.1 assessment and the cefr (1)
 
Assessing learning in Instructional Design
Assessing learning in Instructional DesignAssessing learning in Instructional Design
Assessing learning in Instructional Design
 
7.1 assessment and the cefr (1)
7.1 assessment and the cefr (1)7.1 assessment and the cefr (1)
7.1 assessment and the cefr (1)
 
research-instruments (1).pptx
research-instruments (1).pptxresearch-instruments (1).pptx
research-instruments (1).pptx
 
Rep
RepRep
Rep
 
JC-16-23June2021-rel-val.pptx
JC-16-23June2021-rel-val.pptxJC-16-23June2021-rel-val.pptx
JC-16-23June2021-rel-val.pptx
 
Test characteristics
Test characteristicsTest characteristics
Test characteristics
 
EM&E.pptx
EM&E.pptxEM&E.pptx
EM&E.pptx
 
Qualities of a Good Test
Qualities of a Good TestQualities of a Good Test
Qualities of a Good Test
 
Validity, reliability and feasibility
Validity, reliability and feasibilityValidity, reliability and feasibility
Validity, reliability and feasibility
 
Evaluation in Education
Evaluation in EducationEvaluation in Education
Evaluation in Education
 
Validity and objectivity of tests
Validity and objectivity of testsValidity and objectivity of tests
Validity and objectivity of tests
 
Validity
ValidityValidity
Validity
 
Measurement and scaling
Measurement and scalingMeasurement and scaling
Measurement and scaling
 
Valiadity and reliability- Language testing
Valiadity and reliability- Language testingValiadity and reliability- Language testing
Valiadity and reliability- Language testing
 
D8 and d9 personality test development 10 2007-posting
D8 and d9 personality test development 10 2007-postingD8 and d9 personality test development 10 2007-posting
D8 and d9 personality test development 10 2007-posting
 
Validity, Reliability and Feasibility
Validity, Reliability and FeasibilityValidity, Reliability and Feasibility
Validity, Reliability and Feasibility
 

Recently uploaded

Sugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdf
Sugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdfSugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdf
Sugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdfDharma Homoeopathy
 
Integrated Mother and Neonate Childwood Illness Health Care
Integrated Mother and Neonate Childwood Illness  Health CareIntegrated Mother and Neonate Childwood Illness  Health Care
Integrated Mother and Neonate Childwood Illness Health CareASKatoch1
 
Management of heart failure 23.02.24.pptx
Management of heart failure 23.02.24.pptxManagement of heart failure 23.02.24.pptx
Management of heart failure 23.02.24.pptxchristina388422
 
Prosthesis upper limb and lower limb.pptx
Prosthesis upper limb and lower limb.pptxProsthesis upper limb and lower limb.pptx
Prosthesis upper limb and lower limb.pptxBadalverma11
 
About Dr Ranjit Jagtap daughter's - Aditi & Poulami Jagtap
About Dr Ranjit Jagtap daughter's - Aditi & Poulami JagtapAbout Dr Ranjit Jagtap daughter's - Aditi & Poulami Jagtap
About Dr Ranjit Jagtap daughter's - Aditi & Poulami JagtapAditi Jagtap Pune
 
Homeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdf
Homeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdfHomeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdf
Homeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdfDharma Homoeopathy
 
Best Way 30-Days Keto Meal Plan For Diet
Best Way 30-Days Keto Meal Plan For DietBest Way 30-Days Keto Meal Plan For Diet
Best Way 30-Days Keto Meal Plan For DietGunaPerumal1
 
Breaking Down Oppositional Defiant Disorder Treatments
Breaking Down Oppositional Defiant Disorder TreatmentsBreaking Down Oppositional Defiant Disorder Treatments
Breaking Down Oppositional Defiant Disorder TreatmentsOppositional Defiant Disorder
 
Nose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptx
Nose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptxNose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptx
Nose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptxDr. Rabia Inam Gandapore
 
CHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdf
CHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdfCHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdf
CHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdfSachin Sharma
 
Improve Patient Care with Medical Record Abstraction
Improve Patient Care with Medical Record AbstractionImprove Patient Care with Medical Record Abstraction
Improve Patient Care with Medical Record AbstractionCardiac Registry Support
 
Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...
Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...
Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...aunty1x1
 
ASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptx
ASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptxASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptx
ASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptxAnushriSrivastav
 
Master the Art of Yoga with Joga Yoga Training
Master the Art of Yoga with Joga Yoga TrainingMaster the Art of Yoga with Joga Yoga Training
Master the Art of Yoga with Joga Yoga TrainingJoga Yoga Training
 
Valle Egypt Illustrates Consequences of Financial Elder Abuse
Valle Egypt Illustrates Consequences of Financial Elder AbuseValle Egypt Illustrates Consequences of Financial Elder Abuse
Valle Egypt Illustrates Consequences of Financial Elder AbuseKristin Hetzer
 
Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\DanielOliver74
 
Optimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptx
Optimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptxOptimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptx
Optimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptxLovina Kapoor
 
Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...
Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...
Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...saimasadaf14
 
Importance of Diet on Dental Health.docx
Importance of Diet on Dental Health.docxImportance of Diet on Dental Health.docx
Importance of Diet on Dental Health.docxSachin Mittal
 

Recently uploaded (20)

Sugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdf
Sugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdfSugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdf
Sugar Medicine_ Natural Homeopathy Remedies for Blood Sugar Management.pdf
 
Integrated Mother and Neonate Childwood Illness Health Care
Integrated Mother and Neonate Childwood Illness  Health CareIntegrated Mother and Neonate Childwood Illness  Health Care
Integrated Mother and Neonate Childwood Illness Health Care
 
Management of heart failure 23.02.24.pptx
Management of heart failure 23.02.24.pptxManagement of heart failure 23.02.24.pptx
Management of heart failure 23.02.24.pptx
 
Prosthesis upper limb and lower limb.pptx
Prosthesis upper limb and lower limb.pptxProsthesis upper limb and lower limb.pptx
Prosthesis upper limb and lower limb.pptx
 
About Dr Ranjit Jagtap daughter's - Aditi & Poulami Jagtap
About Dr Ranjit Jagtap daughter's - Aditi & Poulami JagtapAbout Dr Ranjit Jagtap daughter's - Aditi & Poulami Jagtap
About Dr Ranjit Jagtap daughter's - Aditi & Poulami Jagtap
 
Homeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdf
Homeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdfHomeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdf
Homeopathy Medicine for Diabetes_ Balancing Blood Sugar .pdf
 
Best Way 30-Days Keto Meal Plan For Diet
Best Way 30-Days Keto Meal Plan For DietBest Way 30-Days Keto Meal Plan For Diet
Best Way 30-Days Keto Meal Plan For Diet
 
Breaking Down Oppositional Defiant Disorder Treatments
Breaking Down Oppositional Defiant Disorder TreatmentsBreaking Down Oppositional Defiant Disorder Treatments
Breaking Down Oppositional Defiant Disorder Treatments
 
Nose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptx
Nose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptxNose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptx
Nose-Nasal Cavity & Paranasal Sinuses BY Dr.Rabia Inam Gandapore.pptx
 
CHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdf
CHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdfCHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdf
CHAPTER- 1 SEMESTER - V NATIONAL HEALTH PROGRAMME RELATED TO CHILD.pdf
 
Improve Patient Care with Medical Record Abstraction
Improve Patient Care with Medical Record AbstractionImprove Patient Care with Medical Record Abstraction
Improve Patient Care with Medical Record Abstraction
 
Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...
Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...
Contact mE 👙👨‍❤️‍👨 (89O1183OO2) 💘ℂall Girls In MOHALI By MOHALI 💘ESCORTS GIRL...
 
ASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptx
ASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptxASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptx
ASSISTING WITH THE USE OF BED PAN BY ANUSHRI SRIVASTAVA.pptx
 
Master the Art of Yoga with Joga Yoga Training
Master the Art of Yoga with Joga Yoga TrainingMaster the Art of Yoga with Joga Yoga Training
Master the Art of Yoga with Joga Yoga Training
 
Valle Egypt Illustrates Consequences of Financial Elder Abuse
Valle Egypt Illustrates Consequences of Financial Elder AbuseValle Egypt Illustrates Consequences of Financial Elder Abuse
Valle Egypt Illustrates Consequences of Financial Elder Abuse
 
Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
Production.pptx\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
 
Optimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptx
Optimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptxOptimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptx
Optimizing Outcomes Robolens Knee Resurfacing in Chandigarh.pptx
 
Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...
Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...
Jesse Jhaj: Building Relationships with Patients as a Doctor or Healthcare Wo...
 
Importance of Diet on Dental Health.docx
Importance of Diet on Dental Health.docxImportance of Diet on Dental Health.docx
Importance of Diet on Dental Health.docx
 
Importance-of-Protein-During-Pregnancy Time
Importance-of-Protein-During-Pregnancy TimeImportance-of-Protein-During-Pregnancy Time
Importance-of-Protein-During-Pregnancy Time
 

Validity and reliability of questionnaires

  • 2. CONTENTS  Introduction  Steps in questionnaire designing  Validity  Concept of validity  Types of validity  Steps in questionnaire validation  Reliability  Types and measurement of reliability  Conclusion  References
  • 3. INTRODUCTION  Questionnaire: Important method of data collection used extensively  Advantages of questionnaire  Less expensive  Offers greater anonymity  Disadvantages  Application is limited  Response rate is low  Opportunities to clarify issues is lacking
  • 4.  Ideal requisites of a questionnaire:  Should be clear and easy to understand  Layout is easy to read and pleasant to eye  Sequence of questions easy to follow  Should be developed in an interactive style  Sensitive questions must be worded exactly  NOTE: The terminologies research instrument, measuring instrument,scale and test in various parts of the seminar represent questionnaire in this context . . . And item represents each question in a questionnaire
  • 7. The concept of validity  Validity is the ability of an instrument to measure what it is intended to measure.  Degree to which the researcher has measured what he has set out to measure (Smith, 1991)  Are we measuring what we think we are measuring? (Kerlinger, 1973)  Extent to which an empirical measure adequately reflects the real meaning of the concept under consideration (Babbie, 1989)
  • 8. Why validity ?  Validity is done mainly to answer the following questions:  Is the research investigation providing answers to the research questions for which it was undertaken?  If so, is it providing these answers using appropriate methods and procedures?
  • 9. Questions to ponder Investigator Readers of report Experts in the field Logic Statistical tests
  • 10. Logical thinking  Justification of each question in relation to objective of study  Easy if questions relate to tangible matters  Difficult in situations where we are measuring attitude, effectiveness of a program, satisfaction etc  Everybody’s logic doesn’t match . . No statistical backing
  • 11. Statistical procedures  By calculating coefficient of correlations between questions and outcome variables
  • 12. Types of validity Validity Content validity Face validity Criterion related Concurrent Predictive Construct validity
  • 13. CONTENT VALIDITY  Uses logical reasoning and hence easy to apply  Extent to which a measuring instrument covers a representative sample of the domain of the aspects measured  Whether items and questions cover the full range of the issues or problem being measured
  • 14. FACE VALIDITY  The extent to which a measuring instrument appears valid on its surface  Each question or item on the research instrument must have a logical link with the objective
  • 15. Face validity is not content validity. Why?  Face validity  Simply addresses whether a measuring instrument looks valid  Not a validity in technical sense because it does not refer to what is actually being measured rather what it appears to measure  It has more to do with rapport and public relations than with actual validity
  • 16. Other aspects of content validity  Coverage of issue should be balanced  Each aspect should have similar and adequate representation in questions
  • 17. Problems associated with content validity  Based on subjective logic; no definitive conclusion can be drawn or consensus reached  Extent to which questions reflect the objectives of the study may differ. If wordings changed or question substituted, magnitude of link changes
  • 18. CRITERION VALIDITY  The extent to which a measuring instrument accurately predicts behaviour or ability in a given area.  The measuring instrument is called ‘criteria’  It is of two types:  Predictive validity  Concurrent validity
  • 19. Predictive validity  If the test is used to predict future performance  Eg: Entrance exam . . . . Performance of these tests correlates with later performance in professional college  Eg: Written driving test  Eg: measurement of sugar exposure for caries development
  • 20. Concurrent validity  If the test is used to estimate present performance or person’s ability at the present time not attempting to predict future outcomes  Professional college exam  Eg: driving test, pilot test  Eg: measurement of DMFT for caries experience
  • 21. Problems in criterion validity  Cannot be used in all circumstances  Esp in social sciences where some conditions do not have a relevant criteria  Eg: for measuring self-esteem, no criteria can be applied
  • 22. CONSTRUCT VALIDITY  Most important type of validity  Assesses the extent to which a measuring instrument accurately measures a theoretical construct it is designed to measure  Measured by correlating performance on the test with performance on a test for which construct validity has been determined  Eg: a new index for measuring caries can be validated by comparing its values with a standard index (like DMFT)
  • 23.  Another method is to show that scores of the new test differs across people with different levels of outcomes being measured  Eg: Establishing the validity of a new caries index by applying it to different stages of dental caries and calculating its accuracy
  • 24. Summary of Validity CONTENT CRITERION CONSTRUCT CONCURRENT PREDICTIVE What it measures Whether the test covers a representative sample of the domains to be measured The ability of the test to estimate present performance The ability of the test to predict future performance The extent to which the instrument measures a theoretical construct How it is accomplished Ask experts to assess the test to establish that the items are representative of the outcome Correlate performance on the test with a concurrent behaviour Correlate performance on the test with a behaviour in future Correlate performance on the instrument with a performance on an established instrument
  • 26. FACE VALIDITY  Evaluate in terms of: Readability Layout and style Clarity of wording Feasibility
  • 27. CONTENT VALIDITY Two phases Researcher: Conceptualization and domain analysis Experts: Enhancement of content of questionnaire (Seven or more experts) Specify the full domain of content that is relevant to the issue Sample specific areas form this domain Put items/questions in a form that is testable
  • 28. How do experts evaluate validity  Method 1: Average Congruency Percentage (ACP) [Popham, 1978]  Experts compute the percentage of questions deemed to be relevant for them  Take the average of all experts  If the value is > 90 . . . Valid  Eg: 2 experts . . (Expert 1-100%, Expert 2-80%)  Then ACP = 90%
  • 29.  Method 2: Content validity index [Martuza 1977]  Content validity Index for individual items (I-CVI)  Content Validity Index for the scale (S-CVI)
  • 30. I-CVI  Panel of content experts asked to review the relevance of each question on a 4-point Likert scale (minimum 3 maximum 10 experts)  1= not relevant  2= somewhat relevant  3= relevant  4= very relevant  Then for each question, number of experts giving 3 or 4 score is counted (3,4 – relevant; 1,2 – nonrelevant)  Proportion is calculated  Eg: If 4/5 experts give score 3 or 4: I-CVI = 0.80
  • 31. Critics of I-CVI  Collapses experts multipoint assessment into two categories (relevant and non-relevant)  Does not give inference on comprehensiveness of whole questionnaire  Problem of chance agreement. To overcome that, Lynn proposed  Five or fewer experts: all must agree (I-CVI = 1.0)  Six or more: (I-CVI should not be less than 0.78)
  • 32. S-CVI  The proportion of items on an instrument that achieved a rating of 3 or 4 by all the content experts  Two approaches:  S-CVI/UA – Universal agreement  S-CVI/Ave - Average
  • 33.  Which would be an effective measure here ??  S-CVI/UA or S-CVI/Ave
  • 34.
  • 35.  Which to follow?  Report both the values I-CVI and S-CVI rather than using CVI as an acronym  Report the range of I-CVI values  The best method is S-CVI/UA for stringent validity, but will be difficult to use if multiple experts are validating. . In such situations S-CVI/Ave is used
  • 36. CONSTRUCT VALIDITY  Method: Factor analysis  To examine empirically the interrelationship among items and to identify clusters of items that share sufficient variation to justify their existence as a factor or construct to be measured by the instrument  Various items are gathered into common factors  Common factors are synthesized into fewer factors and then relation between each item and factor is measured  Unrelated items are eliminated
  • 38. RELIABILITY  Definition: It is the ability of an instrument to create reproducible results  Each time it is used, similar scores should be obtained  A questionnaire is said to be reliable if we get same/similar answers repeatedly  Though it cannot be calculated exactly, it can be measured by estimating correlation coefficients
  • 39. Reliability measured in aspects of: • Done to ensure that same results are obtained when used consecutively for two or more times • Test-retest method is used STABILITY • To ensure all subparts of a instrument measure the same characteristic (Homogeneity) • Split-half method INTERNAL CONSISTENCY • Used when two observers study a single phenomenon simulataneously • Inter-rater reliability EQUIVALENCE
  • 40. Test-Retest reliability (for stability)  Test administered twice to the same participant at different times  Used for things that are stable over time  Easy and straight-forward approach  Useful for questionnaires, checklist, rating scales etc  Disadvantages  Practice effect (mainly for tests)  Too short intervals in between (effect of memory)  Some traits may change with time
  • 41. Statistical calculation  Administration of instrument to a sample on two different occasions  Scores compared and calculated by using correlation coefficient formula (pearson)
  • 42. Correlation coefficient  Measures the degree of relationship between two sets of scores  Can range from -1 to +1  0 indicates absence of any relationships Correlation coefficient Strength of relationship +/- 0.7 to 1.0 Strong +/- 0.3 to 0.69 Moderate +/- 0.0 to 0.29 None to weak
  • 43. Split halves reliability (homogenity)  Split the contents of the questionnaire into two equivalent halves; either odd/even number or first/second half  Correlate scores of one half with scores of the other  Formula: r = Σ (x-x’)(y-y’) √ Σ(x-x’)2 (y-y’)2  But this r is only for the half, so to check reliability of entire test, use the formula
  • 44.  R’ = 2r/1+r  (r = coefficient of split half, R’ = coefficient of entire test)  Cronbach’s alpha:  Another method of calculation using the formula: R = k/k-1 (1-Σσ1 2/σy2) k = total number of items in list σ1 = variance of individual items σy2 = variance of total test scores
  • 45. Inter-rater reliability (Equivalence)  Used when a single event is measured simultaneously and independently by two or more trained observers R = Number of agreements Number of agreements + Number of disagreements
  • 46. Summary of Reliability TEST RETEST SPLIT HALF INTERRATER What it measures Stability over time Equivalency of items Agreement between raters How it is accomplished Administer the same test to the same people at two different times Correlate performance for a group of people on two equivalent halves of same test Have multiple researchers measure same instrument and determine percentage of agreement between them
  • 47. Conclusion  Validated questionnaire  It is one which has undergone a validation procedure to show that it accurately measures what it aims to, regardless of who responds, when they respond, and to whom they respond or when self-administered and whose reliability has also been examined thereby:  Reducing bias and ambiguities  Better quality of data and credible information
  • 48. In a nutshell . . . . A questionnaire can be reliable but invalid . . . But a valid questionnaire is always reliable . . .
  • 49. Acknowledgements  Dr. Joe Joseph  Dr. Chandrashekar
  • 50. References  Linda Del Greco, Walop W, Richard H McCarthy. Questionnaire development: 2. Validity and Reliability. CMAJ. 1987;136:699–700.  Sushil S, Verma N. Questionnaire validation made easy. Eur J Sci Res. 2010;46(2):172–8.  Polit DF, Cheryl Tatano Beck. The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Res Nurs Health. 2006;29:489–97.  Reliability and Validity Module 6. Cengage Learning; 2010.
  • 51.  Rama B Radhakrishna. Tips for Developing and Testing Questionnaires/Instruments. J Ext. 2007;35(1):710–4.  06Article04.pdf [Internet]. [cited 2015 Apr 7]. Available from: http://www.uk.sagepub.com/salkind2study/articles/06Article04.pdf  pta_6871_6791004_64131.pdf [Internet]. [cited 2015 Apr 7]. Available from: http://cfd.ntunhs.edu.tw/ezfiles/6/1006/attach/33/pta_6871_6791004_6 4131.pdf  Questionnaire designing and validation [Internet]. [cited 2015 Apr 7]. Available from: http://www.jpma.org.pk/full_article_text.php?article_id=3414
  • 52.  Suresh K Sharma. Nursing Research and Statistics. 1st ed. New Delhi: Elsevier Saunders;  Edward G, Richard Zeller. Reliability and Validity Assessment. New Delhi: SAGE publication; 1979.  Ranjit Kumar. Research Methodology - A step by step guide for beginners. 3rd ed. New Delhi: SAGE publication; 2012.  Articles from Dr. Joe