This document provides an overview of cross-cultural adaptation and validation of health measures for use across cultures. It discusses the need to adapt measures for use in languages other than the source language. The key steps in cross-cultural adaptation are outlined, including translation, back-translation, expert committee review, pre-testing, and documentation. Validity aspects covered include face validity, content validity, construct validity, and factorial validity. Reliability is also mentioned. The document uses examples and outlines standards for ensuring measures maintain equivalence across cultures.

1. DR.SHABNA.G.S
JUNIOR RESIDENT
GOVERNMENT DENTAL COLLEGE, KOTTAYAM
1

2. CONTENTS
• INTRODUCTION
• CROSS-CULTURAL ADAPTATION
• VALIDITY
FACE VALIDITY
CONTENT VALIDITY
CONSTRUCT VALIDITY
FACTORIAL VALIDITY
• RELIABILITY
• CONCLUSION
2

3. INTRODUCTION
3

4. 4

5. HEALTH MEASURES
HEALTH MEASURES- provide information on the health of a population.
5

6. INTRODUCTION
Majority of health related measures are in English language
6

7. NEED TO ADAPT HEALTH MEASURES FOR USE IN OTHER
THAN THE SOURCE LANGUAGE
MULTINATIONAL
MULTICULTURAL
MULTIREGIONAL SURVEYS
GOAL - TO PRODUCE COMPARABLE MEASURES ACROSS
MULTINATIONAL, MULTICULTURAL, OR MULTIREGIONAL
POPULATIONS
7

8. CROSS-CULTURAL ADAPTATION
• To explore the same question in several cultures or measure differences across cultures
• To achieve equivalence between the original and adapted questionnaire
8

9. IF MEASURES ARE TO BE USED ACROSS CULTURES
• Items must be well linguistically
• Items must be culturally
9

10. CROSS-CULTURAL ADAPTATION
• Process of producing a document in
the target language from a source
version
TRANSLATION ADAPTATION
• Process of considering any differences
between the source and the target
culture to maintain equivalence in
meaning
10

11. CROSS-CULTURAL ADAPTATION
PROBLEMS
• Linguistic problems during translation
• Item can have a very different meaning or no meaning at all in a specific
cultural background
11

12. Health Assessment Questionnaire(HAQ)
12

13. • Thai version of the HAQ
Thai culture- people do not use bathtub
NEW ITEM - Sitting to pay homage to a sacred image
13

14. 14

15. CROSS-CULTURAL ADAPTATION OF HRQOL INSTRUMENTS
• based on criteria’s set by Guillemin et al
Stage I
TRANSLATION
Stage II
SYNTHESIS
Stage V
PRETESTING
Stage IV
EXPERT COMMITTEE
REVIEW
Stage III
BACK
TRANSLATION
TWO TRANSLATIONS[TI &T2] SYNTHESIZED T1 &T2
INTO T-12
CREATED 2 BACK
TRANSLATION OF T-12
VERSION
REVIEW ALL REPORTS
PRODUCE PRE-FINAL
VERSION
PRETESTED IN 20
Stage VI-SUBMISSION AND APPRASIAL OF ALL WRITTEN REPORTS BY DEVELOPERS
Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: Literature review and proposed guidelines. J Clin Epidemiol 1993;46:1417-32.15

16. STAGE I
TRANSLATION
• Forward translation
• 2 independent Bilingual translators whose mother tongue is the target
language
TRANSLATOR I
Aware of the concepts being examined
Intended to provided equivalency from a
clinical perspective as well as from a
measurement perspective
TRANSLATOR II- NAIVE TRANSLATOR
Neither be aware nor informed of the
concepts being quantified
Preferably should have no medical or
clinical background
More likely to detect different meaning of
the original than the first translator
16

17. STAGE II
SYNTHESIS OF THE TRANSLATIONS
BY USING
• Original questionnaire
• First translator’s (T1) version
• Second translator’s (T2) version
SYNTHESIS (producing one common translation T-12)
17

18. STAGE III
BACK TRANSLATIONS
• The back-translations (BT1 and BT2) are produced by two persons with the source
language (English) as mother tongue
• Neither be aware nor be informed of the concepts explored
• Should preferably be without medical background
• Agreement between the back translation and the original source version-
TRANSLATIONAL VALIDITY
18

19. STAGE IV
EXPERT COMMITTEE
COMPOSITION
• Methodologists
• Health professionals
• Language professionals
• Translators(forward and back translators)
19

20. MATERIAL AT THE DISPOSAL OF THE COMMITTEE
• Original questionnaire
• Each translation (T1, T2, T12, BT1, BT2)
• Corresponding written reports
REVIEW ALL THE TRANSLATIONS AND REACH A CONSENSUS ON ANY DISCREPANCY
20

21. ROLE
• Consolidate all the versions of the questionnaire
• Develop the prefinal version of the questionnaire for field testing
21

22. SEMANTIC EQUIVALENCE
• Do the words mean the same thing?
• Are their multiple meanings to a given item?
• Are there grammatical difficulties in the translation?
IDIOMATIC EQUIVALENCE
• Colloquialisms, or idioms, are difficult to translate
• The committee may have to formulate an equivalent expression in the target version
For example the term “feeling downhearted and blue” from the SF-36
22

23. 23

24. EXPERIENTIAL EQUIVALENCE
• Experience of daily life capturing items may not be suitable in a different country or culture
• Do you have difficulty eating with a fork?
[when that was not the utensil used for eating in the target country]
CONCEPTUAL EQUIVALENCE
Often words hold different conceptual meaning between cultures –
24

25. STAGE-V
TESTINGPREFINALVERSION
• Pretest OF PREFINAL VERSION on 20 -40 persons
• Each subject
• Completes the questionnaire
• Interviewed to probe about what he or she thought was meant by
each questionnaire item and chosen response
25

26. STAGE-V
TESTINGPREFINALVERSION
• Both the meaning of the items and responses would be explored
• Look for a high proportion of missing items or single responses.
26

27. STAGE-VI
SUBMISSION OF DOCUMENTATION
• A process audit to verify all the steps followed
• Verify that the recommended stages were followed
27

28. CONSTRUCTS
• Constructs are mental abstractions used to express the ideas, people,
organizations, events and/or objects/things that we are interested in
• Consists of several attributes
• Evaluated by a number of selected items
28

29. • Ideas - Racism, self-esteem, poverty, trust, morality, tolerance, air pollution, genetic engineering
• People - Age, gender, ethnicity, height, obesity, morbidity, energy, muscle soreness, fatigue
• Organisations - Financial performance, corporate social responsibility, organisational culture
• Events - famine, urban regeneration, secularism
• Objects/Things - Sun, trees, flowers, amino acids, stem cells
29

30. CONSTRUCTS
• Measured by composite measurement scales (CMSs)
• A CMS -items or questions that assess one or several attributes scored by a
scale
30

31. FACE VALIDITY
• SURFACE VALIDITY ,APPEARANCE VALIDITY, LOGICAL VALIDITY
• a subjective, superficial assessment of whether the measurement procedure you use in a
study appears to be a valid measure of a given variable or construct
• an inferior form of validity
• Do participants agree with items and wording of them in an instrument to realize research
objectives?
31

32. EXAMPLE
• A researcher wants to identify the best football
players in the league
Administer a questionnaire-How many times have you played in the team this year?
Participants thinks that they are completing a questionnaire to
identify the best football players in the league
STRONG FACE VALIDITY
32

33. CONTENT VALIDITY
• Definition validity ,Logical validity
• The extent to which the elements within a measurement procedure are relevant and
representative of the construct that they will be used to measure (Haynes et al., 1995)
33

34. Provides the preliminary evidence on construct validity of an instrument
If an instrument lacks content validity-impossible to establish RELIABILITY
Provide information on the representativeness and clarity of items and help improve
an instrument through achieving recommendations from an EXPERT PANEL
34

35. EXPERT PANEL consists of content experts and lay experts
LAY EXPERTS are the potential research subjects
CONTENT EXPERTS are professionals who have research experience or work in the field
Subjects of the target group as expert – Adequate representation of population for
whom the instrument is being developed
35

36. • QUALITATIVE CONTENT VALIDITY METHOD
• CONTENT EXPERTS AND TARGET GROUP’S RECOMMENDATIONS
Observe grammar
Use appropriate and correct words
Apply correct and proper order of words in items
Appropriate scoring
36

37. • QUANTITATIVE CONTENT VALIDITY
Quantified by
Content Validity Ratio (CVR)
Content Validity Index(CVI)
37

38. CONTENT VALIDITY RATIO (CVR)
• The CVR (content validity ratio) proposed by Lawshe (1975) -How
many “experts” within a panel rate an item “essential”
38

39. • Experts are requested to score each item from 1 to 3
• Rate items into one of three categories:
“not necessary, useful but not essential, essential” is used
FORMULA
• CVR=(Ne - N/2)/(N/2)
Ne =number of panelists indicating "essential"
N =total number of panelists
39

40. • Content validity ratio varies between 1 and -1
• The higher score indicates -agreement of members of panel on the
necessity of an item in an instrument
40

41. • The most widely reported approach for content validity - CONTENT
VALIDITY INDEX
• A panel of subject experts rate each item based on relevance as “most
relevant,” “relevant,” “can be avoided,” and “not relevant”
• The expert rating was quantified as a CONTENT VALIDITY INDEX (CVI)
• Items with CVI < 0.8 were deleted
41

42. • For relevancy, content validity index can be calculated both for item
level (I-CVIs) and the scale-level (S-CVI)
ICVI=
Number of experts giving a rating 3 or 4 to the relevancy of each item
Total number of experts
42

43. • S-CVI
Number of ITEMS considering a rating 3 or 4 to the relevancy of each item
Total number of ITEMS
43

44. • The I-CVI expresses the proportion of agreement on the relevancy of each
item, which is between zero and one
• SCVI is defined as “the proportion of items on an instrument that achieved
a rating of 3 or 4 by the content experts”
44

45. S-CVI
 Two approaches:
 S-CVI/UA– Universal agreement
 S-CVI/Ave - Average
45

46. 46

47. CONSTRUCT VALIDITY
• Construct validity is the degree to which an instrument measures the
trait or theoretical construct that it is intended to measure
• It is the most valuable and most difficult measure of validity
• It is a measure of how meaningful the scale or instrument is when it is
in practical use
47

48. Convergent validity
• Same concept measured in different ways yields similar results
• In convergent validity where different measures of the same concept
yield similar results, a researcher uses self-report versus observation
48

49. • Since the study used two different measurement procedures, how
confident can we be that both measurement procedures were
measuring the same construct (i.e., post-natal depression)
• If both measurement procedures were new -convergent validity
• If one was new (e.g., the 10-question survey), but the other was well-
established (e.g., the participant observation scale)-concurrent validity
49

50. • Discriminant validity-
the extent to which traits are distinct (Carmines & Zeller,1979)
ability of tool to differentiate between diseased and healthy
50

51. FACTORIAL VALIDITY
• This is an empirical extension of content validity
• CONSTRUCT OF INTEREST many dimensions different domains of
a general attribute
• Several items put up to measure a particular dimension within a construct
of interest is supposed to be highly related to one another than those
measuring other dimensions
51

52. • For instance, using health-related quality of life questionnaire using short
form - 36 version 2 (SF-36v2)
• This tool has 8 dimensions
• All the items of SF-36v2 questionnaire measuring social function (SF),
which is one of the 8 dimension, should be highly related than those items
measuring mental health domain which measure another dimension
52

53. Charles Edward Spearmen was known for his seminal work
on testing and measuring of HUMAN INELLIGENCE by
using the FACTOR ANALYSIS during World WarI.
CHARLES EDWARD SPEARMEN
(BRITISH PSYCHOLOGIST)
53

54.  A factor is a linear combination of variables
 It is a construct that is not directly observed
 Needs to be inferred from the input variables
 Factor analysis is a collection of methods used to examine how underlying
constructs influence the responses on a number of measured variables
54

55. • VARIABLE REDUCTION TECHNIQUE
• Two types of factor analysis
 Exploratory
 Confirmatory
FACTOR ANALYSIS
55

56.  Exploratory factor analysis (EFA) -discover the nature of the constructs influencing a set of
responses
 Confirmatory factor analysis (CFA) tests whether a specified set of constructs is
influencing responses in a predicted way
56

57. COMMON FACTOR MODEL
57

58. • Performed by examining the pattern of correlations (or covariances)
between the observed measures
• Measures that are highly correlated (either positively or negatively) -
influenced by the same factors
• Measures that are relatively uncorrelated -influenced by different factors
58

59. EXPLORATORY FACTOR ANALYSIS
OBJECTIVES
The number of common factors influencing a set of measures.
The strength of the relationship between each factor and each observed
measure
59

60. SAMPLE SIZE IN FACTOR ANALYSIS
• Concept - N/p [item to participant ratio of 1:10]
• Each question -addressed by 10 respondents
[1978]-recommends 1 to 10 ratio
[2007]-recommends 1 to 5 ratio
60

61. ASSESSMENT OF FACTORABILITY OF THE DATA
• TWO METHODS
Bartlett’s test of sphericity (Bartlett 1954)
Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (Kaiser 1970, 1974)
61

62. BARTLETT’S TEST OF SPHERICITY (BARTLETT 1954)
• Bartlett’s Test of Sphericity compares an observed correlation matrix to the
identity matrix
• NULL HYPOTHESIS -variables are orthogonal [not correlated]
• ALTERNATIVE HYPOTHESIS -variables are not orthogonal [they are correlated
enough -correlation matrix diverges significantly from the identity matrix]
62

63. Correlation Matrix vs. Identity Matrix
• A correlation matrix -the correlation coefficients between variables
• An identity matrix is a matrix in which all of the values along the
diagonal are 1 and all of the other values are 0
63

64.  If the numbers in this matrix represent correlation coefficients it means that each variable is perfectly
orthogonal (i.e. “uncorrelated”) to every other variable and thus a data reduction technique like PCA or factor
analysis would not be able to “compress” the data in any meaningful way
64

65. • Bartlett’s test of sphericity should be significant (p < .05) for the factor
analysis to be considered appropriate
Bartlett’s Test of Sphericity -the correlation matrix of the variables in our dataset
diverges significantly from the identity matrix
65

66. • The KMO index ranges from 0 to 1 [0.6 -minimum value for a good factor
analysis]
 The larger the value of KMO- more adequate sample for running the factor analysis
 Kaiser recommends accepting values greater than 0.5 as acceptable
KAISER-MEYER-OLKIN (KMO) MEASURE OF SAMPLING ADEQUACY
66

67.  Analyses the pattern of correlations between variables in the correlation matrix
 Which variables tend to correlate highlytogether?
 If variables are highly correlated-they represent the same underlying dimension
 Factor analysis pinpoints the clusters of high correlations between variables and for
each cluster, it will assign a factor
67

68. THE SAQ (SPSS ANXIETY QUESTIONNAIRE)
 Statistics makes me cry
 My friends will think I’m stupid for not being able to cope with SPSS
 Standard deviations excite me
 I dream that Pearson is attacking me with correlation coefficients
 I don’t understand statistics
 I have little experience of computers
 All computers hate me
 I have never been good at mathematics
EXAMPLE
68

69. EXAMPLE
69

70. 70

71. Most items have some correlation with each other ranging from
• r=−0.382 for Items 3 and 7
• r=.514 for Items 6 and 7
RELATIVELY HIGH CORRELATIONS
GOOD CANDIDATE FOR FACTOR ANALYSIS
71

72. PARTITIONING THE VARIANCE IN FACTOR ANALYSIS
Factor analysis assumes that variance can be partitioned into two types of variance
 COMMON
 UNIQUE
72

73. PARTITIONING THE VARIANCE IN FACTOR ANALYSIS
73

74.  Common variance
• Variance in a variable that is shared with other variables
• Items that are highly correlated will share a lot of variance
• COMMUNALITY
 The proportion of a variable's variance explained by the extracted factor structure
 Final communality estimates are the sum of squared loadings for a variable in an
orthogonal factor matrix
74

75. •Unique variance is any portion of variance that’s not common. There are two types:
Specific variance: variance that is specific to a particular item
e.g., Item 4 “All computers hate me”may have variance that is attributable to anxiety
about computers in addition to anxiety about SPSS
Error variance: comes from errors of measurement and basically anything unexplained by
common or specific variance
75

76. 76

77. FACTOR EXTRACTION
Determining the smallest number of factors that can be used to best represent the interrelationships
among the set of variables
APPROACHES
 Principal Components-most commonly used approach
 Principal Factors
 Image Factoring
 Maximum Likelihood Factoring
 Alpha Factoring
 Unweighted Least Squares
 Generalised Least Squares
77

78. PRINCIPAL COMPONENT ANALYSIS
• To create one or more index variables from a larger set of measured
variables
• Using a linear combination of a set of variables
• The created index variables are called components
78

79. 79

80. PRINCIPAL COMPONENT ANALYSIS
ASSUMPTION
 No unique variance
 Total variance is equal to common variance
 If the total variance is 1, then the common variance is equal to the communality
80

81. 81

82. SPSS
First go to Analyze – Dimension Reduction – Factor. Move all the observed variables over the Variables: box to be analyze
82

83. EIGEN VALUES AND EIGEN VECTORS
• Eigenvalues represent the total amount of variance that can be explained by a given
principal component
• Can be positive or negative in theory
• In practice they explain variance which is always positive
83

84. •If eigenvalues are greater than zero - GOOD SIGN
•Eigenvalues close to zero - ITEM MULTICOLLINEARITY
• KAISER’S CRITERION [EIGENVALUE RULE]
Only factors with an eigenvalue of 1.0 or more are retained for further investigation
84

85. • Eigenvectors represent a weight for each eigenvalue
• COMPONENT LOADING[correlation of each item with the principal component]
= eigenvector*square root of the eigenvalue
• Eigenvector associated with Item 1 on the first component = 0.377
• Eigenvalue of Item 1 =3.057
• First component =(0.377)√3.057=0.659[correlation of the first item with the
first component is 0.659]
85

86. COMPONENT MATRIX
86

87.  Shows correlation of each item with the component
 Each item has a loading corresponding to each of the 8 components
 For example
Item 1 is correlated 0.659 with the first component
Item 1 is correlated 0.136 with the second component
Item 1 is correlated −0.398 with the third, and so on.
 The square of each loading represents the proportion of variance explained by a particular component
87

88.  For Item 1, (0.659)2=0.434 or 43.4% of its variance is explained by the first component
 For item 1, (0.136)2=0.018 or 1.8% of the variance is explained by the second component
 The total variance explained by both components is 43.4%+1.8%=45.2%
 If we keep going on adding the squared loadings cumulatively down the components, we find that it sums
to 1 or 100%. This is also known as the COMMUNALITY
 IN PCA THE COMMUNALITY FOR EACH ITEM IS EQUAL TO THE TOTAL VARIANCE
88

89.  Summing the squared component loadings across the components (columns) gives you the
COMMUNALITY ESTIMATES FOR EACH ITEM
 Summing each squared loading down the items (rows) gives you the EIGENVALUE FOR EACH
COMPONENT
 For example
To obtain the first eigenvalue we calculate:
 (0.659)2+(−.300)2–(−0.653)2+(0.720)2+(0.650)2+(0.572)2+(0.718)2+(0.568)2=3.057
89

90. CHOOSING THE NUMBER OF COMPONENTS TO EXTRACT
• Choose components that have eigenvalues greater than 1
• Confirmed by the Scree Plot which plots the eigenvalue by the component
number - CATELL’S SCREE TEST (CATELL 1966)
90

91. 91

92. • Catell recommends
 Retaining all factors above the elbow, or break in the plot
[these factors contribute the most to the explanation of the variance
in the data set]
92

93. RUNNING A PCA WITH 2 COMPONENTS IN SPSS
• The only difference is UNDER Fixed number of factors
Factors to extract -enter 2
93

94. COMMON FACTOR ANALYSIS
PCA COMMON FACTOR ANALYSIS
 Assumes that there common variances
takes up all of total variance
 Assumes that total variance can be
partitioned into common and unique
variance
94

95. CRITERION-RELATED VALIDITY is assessed when one is interested in determining the
relationship of scores on a test to a specific criterion
 A measure of how well questionnaire findings stack up against another instrument or
predictor
 CONCURRENT VALIDITY refers to the extent to which results of a particular
measurement correspond to those of a previously established measurement for the
same construct
95

96. RELIABILITY
 Definition: It is the ability of an instrument to create reproducible results
 Each time it is used, similar scores should be obtained
 Aquestionnaire is said to be reliable if we get same/similar answers repeatedly
96

97. Reliabilitymeasuredin aspectsof:
• Done to ensure that same results are obtained
when used consecutively for two or moretimes
• Test-retest method is used
STABILITY
• Toensure all subparts of a instrument measure
the same characteristic (Homogeneity)
• Split-half method
• Cronbach’s alpha
INTERNAL
CONSISTENCY
97

98. TEST-RETESTRELIABILITY (FORSTABILITY)
 Test administered twice to the same participant at different times
 Used for things that are stable overtime
 Disadvantages
 Too short intervals in between (effect of memory)
 Some traits may change with time
98

99. Statistical calculation
 Administration of instrument to a sample on two different occasions
 Scores compared and calculated byusing correlation coefficient
formula (pearson)
99

100. Split halvesreliability (homogenity)
 Split the contents of the questionnaire into two equivalent halves;
 Correlate scores of one half with scores of the other
100

101.  Cronbach’s alpha:
• Most commonly used to assess the internal consistency of a questionnaire (or survey)
that is made up of multiple Likert-type scales and items
To start the analysis-CLICKING on the Analyze menu, select the Scale option and the
Reliability Analysis sub-option
101

102. 102

103. 103

104. 104

105. 105

106. CLICK on OK.
106

107. 107

108. CONCLUSION
• Poor cross cultural adaptation process may lead to an instrument that is not equivalent to the
original questionnaire
• The lack of equivalence limits the comparability of responses across populations divided by
language or by culture
• It allows data collection efforts to be the same in cross national studies
108

109. BIBLIOGRAPHY
• de Vet HC, Adèr HJ, Terwee CB, Pouwer F. Are factor analytical techniques used appropriately
in the validation of health status questionnaires? A systematic review on the quality of factor
analysis of the SF-36. Qual Life Res 2005;14:1203-18.
• Bolarinwa OA. Principles and methods of validity and reliability testing of questionnaires used
in social and health science researches. Niger Postgrad Med J 2015;22:195-201
• Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health related quality of life
measures: literature review and proposed guidelines. J Clin Epidemiol 1993;46:1417–32
109

110. • Sun J, Won C, Damansara PJ. Questionnaire translation and psychometric properties
evaluation. SEGi Univ Coll 2009;2:45-51.
• Anderson RT, Aaronson N, Wilkin D. Critical review of the international assessments of
health-related quality of life generic instruments. In: The International Assessment of
Health-Related Quality of Life: Theory, Translation, Measurement and Analysis. Oxford, UK:
Rapid Communication of Oxford; 1995:11–37
• Batra M, Aggarwal VP, Shah AF, Gupta M. Validation of Hindi version of oral health impact
profile-14 for adults. J Indian Assoc Public Health Dent 2015;13:469-74.
• Bolarinwa OA. Principles and methods of validity and reliability testing of questionnaires
used in social and health science researches. Niger Postgrad Med J 2015;22:195-201
110

111. • Gandek B, Ware JE Jr, IQOLA Group. Methods for validating and norming
translations of health status questionnaires: the IQOLA project approach. J Clin
Epidemiol 1998;51:953–59.
• Gonzalez-Calvo J, Gonzalez VM, Lorig K. Cultural diversity issues in the
development of valid and reliable measures of health status. Arthritis
Care Res 1997;10:448–56
111

112. THANK YOU
112