Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Cross-cultural adaptation: Translation and Adaptation Issues Influencing the Normative 
Interpretation of Assessment Instr...
Procedures for Adapting Assessment Measures for New Populations 
Anytime a measure is simply used with a population that d...
recommended or some of those steps provided here might not be needed. Furthermore, in some 
instances, specific steps woul...
These steps allowed obtaining conceptual, semantic, idiomatic, experiential and 
operational equivalences, in addition to ...
3. Adapt the draft instrument on the basis of the comments of the reviewers. The 
original test translator or adapter need...
adapted for use with a new population, unless they are used purely as research measures, 
especially those research measur...
It is also appropriate to attempt to determine whether the revised assessment device is being 
misused or misinterpreted b...
References: 
American Educational Research Association, American Psychological Association, & National 
Council on Measure...
Psychological Science. (1993). Standards for adapting instruments and establishing score 
equivalence. Manuscript in prepa...
Upcoming SlideShare
Loading in …5
×

Translation and adaptation issues influencing the normative interpretation of assessment instruments

700 views

Published on

Translation and adaptation issues influencing the normative interpretation of assessment instruments

Published in: Education
  • Be the first to comment

Translation and adaptation issues influencing the normative interpretation of assessment instruments

  1. 1. Cross-cultural adaptation: Translation and Adaptation Issues Influencing the Normative Interpretation of Assessment Instruments The adaptation of assessment instruments for new target populations is generally required when the new target population differs appreciably from the original population with which the assessment device is used in terms of culture or cultural background, country, and language. Most cross-cultural adaptations of assessment instruments involve the translation of an instrument from one language into another (Geisinger, 1994). In some instances, however, adaptations of assessment instruments are needed even when the language remains the same, because the culture or life experiences of those speaking the same language differ. Numerous instruments have been adapted for use with groups that differ experientially from those with whom the instrument was originally used (Geisinger, 1994). Example: The Minnesota Multiphase Personality Inventory-Adolescent [MMPI-A], for example, is an adaptation of the MMPI-2 developed to be used with adolescents, whose life experiences systematically differ from those of adults. Other measures have been developed with different versions for men and women, for example, again on the basis of the generally different life experiences and resultant orientations of men and women (Butcher, 1985; Butcher & Pancheri, 1976). In recent years, the term test adaptation frequently has replaced test translation. This shift in terminology documents the adaptations in references to culture, in content, and in wording that are needed in addition to simple translation in revising a test. These changes are needed to meet the requirements of a circumstance that differs qualitatively from the original use of an assessment device. Few guidelines regarding test adaptations have been developed, and those that do exist have not been widely circulated. The International Test Commission, working in conjunction with the European Association of Psychological Assessment, the International Association of Applied Psychology, the International Association of Cross-Cultural Psychology, the International Association for the Evaluation of Educational Achievement, the International Language Testing Association, and the International Union of Psychological Science, has begun systematizing the procedures that are recommended in test adaptation (International Test Commission, 1993, p. 1).
  2. 2. Procedures for Adapting Assessment Measures for New Populations Anytime a measure is simply used with a population that differs qualitatively from the one for which it was originally developed, one must check its continued validity and usefulness in that new population, even if the test itself remains unchanged. When it is likely that a measure needs to be adapted for use with a new population, the onus is even greater on those adapting the measure to demonstrate its usefulness (Geisinger, 1994). Does a Given Measure Need to Be Adapted? An early decision to be made is whether or not an assessment device actually needs to be adapted for a new intended use. The answer to this question is generally easy when applied to an extreme circumstance. When a new target population is not likely to differ appreciably from the original population with whom the assessment device was used, it is unlikely that the measure needs to be adjusted to that new population. A measure that has been developed with students from Pennsylvania State University would not likely need to be adapted if it were to be administered to students from the University of North Carolina, for example. (However, if it were found that some items on the instrument used local vernacular or were based on information known only to residents of central Pennsylvania, then some minor accommodations would be needed.) At the other extreme, when one wishes to administer a measure to individuals who cannot communicate in the language in which the measure was originally written, then the measure obviously must be translated and perhaps otherwise altered. Indeed, such a translation or adaptation must typically consider cultural as well as language differences between the original and the target populations. Descriptions of adaptations of the MMPI for Turkey, Hong Kong, Greece, and Chile may be found in Savasir and Erol (1990),Cheung (1985), Manos (1985), and Rissetti and Makes (1985), respectively, and serve as excellent examples of both language and cultural adaptations of that particular measure (Geisinger, 1994). Steps for Adapting a Measure The procedures provided later are intended to serve as general guidelines for adapting an assessment instrument to a new culture and language population. Others (e.g., Bracken & Barona, 1991; Butcher, in press-a) have provided lists of the steps suggested for the adaptation of assessment instruments from one language and culture to another. In some cases, all of the steps provided here would be required. In other situations, either additional steps would be
  3. 3. recommended or some of those steps provided here might not be needed. Furthermore, in some instances, specific steps would need to be repeated in an iterative fashion. For example, if validation research failed to yield acceptable evidence supporting the use of the assessment instrument, the adaptation process might begin anew. Alternatively, minor changes might be required, followed by field testing, and so forth. Nevertheless, the steps detailed here provide a test translator with some common approaches to adapting a measure for a new target population(Mininel et al., 2012). Cross-cultural adaptation procedures: The method followed international guidelines for studies of this kind, including the following steps: Initial translation Synthesis of translations Back translation Evaluation by an expert committee Testing of the penultimate version
  4. 4. These steps allowed obtaining conceptual, semantic, idiomatic, experiential and operational equivalences, in addition to content validity (Mininel et al., 2012). 1. Translate and adapt the measure: Sometimes an instrument can be translated or adapted on a question-by-question basis. At other times, it must be adapted and translated only in concept. A personality inventory item that asks an adolescent whether he or she prefers going to watch a movie or to attend a dance will make little sense to an individual from a non-Western, developing country. The choice presented in the item hence must be adapted culturally to be both a meaningful decision and one comparable to the item when responded to by members of the original population. Those translating or adapting an assessment instrument must meet a rigorous set of requirements. They must be fluent in both languages, extremely knowledgeable about both cultures, and expert in both the characteristics and the content measured on the instrument and the uses to which the assessment instrument will be put. Back transition was once the technique of choice in test translation. In this procedure, an original translation would render items from the original version of the instrument to the second language, and a second translator—one not familiar with the instrument—would translate the instrument back into the original language. This second translation would be compared with the original version of the instrument. Unfortunately, it has been found that, when translators knew that their work was going to be subjected to back translation, they would use wording that ensured that a second translation would faithfully reproduce the original version rather than a translation using the optimal wording in the target language (Hambleton, 1993). 2. Review the translated or adapted version of the instrument . Rather than perform a back translation, a more effective technique to ensure that the translation or adaptation has been conducted appropriately is to use a group of individuals meeting the same criteria as the test translator (as described in Step 1) to review carefully the quality of the translation or the adaptation. This editorial review could be accomplished in a group meeting, with individual reviews by the group members (much like the reviews of journal articles), or through some combination thereof. Perhaps the most effective strategy would be to have the individuals composing the panel (a) review the items and react in writing, (b) share their comments with one another, and (c) meet to consider the points made by each other and to reconcile any differences of opinion.
  5. 5. 3. Adapt the draft instrument on the basis of the comments of the reviewers. The original test translator or adapter needs to consider the comments made by the panel of experts described in Step 2. Such deliberations should occur only after the panel has met without the translator or adapter present to discuss their concerns. The translator or adapter can then meet with the panel to explain the reasons for drafting the instrument in the manner used. Similarly, the panel can explain why they reacted to the draft as they did. Through this discursive process, the final instrument will reflect the best judgment of the entire group (Geisinger, 1994). 4. Pilot test the instrument. A few trial administrations of the assessment device should be made simply to learn of the potential problems faced by those responding to the assessment instrument. At this time, a small sample of individuals comparable to the eventual target population should be identified; administered the assessment device; and then interviewed as to the understandability of the instructions, acceptability of the time limits, wording of the items, and so on. The instrument should be changed in light of these early findings (Geisinger, 1994). 5. Field test the instrument. Once pilot testing has been performed and the instrument is revised accordingly, it is ready to be field tested. At this stage, the instrument is administered to a large sample representative of the eventual population to be assessed with the instrument. (This stage can also be performed in several waves, each involving a specific data collection project.) A number of initial analyses should be performed with the data emerging from this data collection. Internal consistency reliability analyses should be performed, for example. If possible, test-retest reliability analyses also should be included in this phase of the instrument adaptation process. Traditional or item-response theory item analyses should be performed. With regard to all of the analytic processes described, comparisons should be made with the corresponding results of the instrument in the original language or culture. It is also possible to compare performance on individual items, in comparison with data collected with the original instrument in the country of its origin. Especially useful with cognitive- type assessments, this set of procedures has been called differential item functioning analyses and was once called item bias detection methods. (The use of these procedures in cross-cultural testing situations may be found in O'Brien, 1992, or van de Vijver & Poortinga, 1991. For a brief discussion of some of the techniques themselves, see Cole & Moss, 1989.) 6. Standardize the scores. If desirable and appropriate, equate them with scores on the original version. At this time, it is sufficient to state that most instruments that are translated or
  6. 6. adapted for use with a new population, unless they are used purely as research measures, especially those research measures that are unpublished, use a standard scale such as the T scale (Mean of 50, standard deviation of 10) to present their scores. Using data from the field testing, if the sample is large enough (probably at least 750-1,000 participants, depending on how many subgroups in the population need adequate representation), it probably is possible to perform a standardization. If the sample is not large enough or fully representative, then collection of a new standardization sample is needed. The analytic procedures involved in such processes are straightforward and provided in Peterson, Kolen, and Hoover (1989). 7. Perform validation research as appropriate. Information on the research needed to document that (a) an assessment device measures the same qualities in both languages or versions and (b) the new version continues to provide scores that are interpretable in the manner proposed (Geisinger, 1994). 8. Develop a manual: and other documents for the users of the assessment device. "Test developers have a responsibility to provide evidence regarding reliability and validity for stated test purposes, as well as manuals and norms, when appropriate, to guide proper interpretation" (American Educational Research Association et al., 1985, p. 25). Just adapting a measure for use in a second setting, no matter how much it might be needed I that setting, is not professionally acceptable. Documents attesting to the value of using the adapted instrument and describing its appropriate use are a required part of the adaptation of any measure for professional use (Geisinger, 1994). 9. Train users. Standard 6.6 of the Standards for Educational and Psychological Testing (American Educational Research Association et al., 1985) states that responsibility for test use should be assumed by or delegated only to those individuals who have the training and experience necessary to handle this responsibility in a professional and technically adequate manner. Any special qualifications for test administration or interpretation noted in the manual should be met (p. 42). 10. Collect reactions from users. As one implements the newly revised instrument, it is frequently propitious to gather comments from those actually using the instrument. Such comments could guide future changes in the instrument and suggest avenues for needed research.
  7. 7. It is also appropriate to attempt to determine whether the revised assessment device is being misused or misinterpreted by users (Geisinger, 1994). What Do Scores on the Adapted Measure Mean? The interpretation of any score on an assessment device is dependent on many kinds of information. Clearly, validation (including information on fairness and bias; see Geisinger, 1992a) and reliability information are critical to test interpretation. Normative information carries meaning to a professional regarding the placement of an individual within a population distribution of test takers, permitting the professional to interpret the likely meaning of the score. This interpretative meaning of test scores is especially useful for tests with well-established validation information and norms (Geisinger, 1994). Example: Thus, a professional admissions counselor at a university can look at an individual's scores on the SAT or a similar measure and approximate the probability that the prospective student will succeed at the university. Similarly, an informed admissions committee making decisions with regard to the applications for graduate study in psychology can consider their applicants' Graduate Record Examination scores by using their accumulated (validation) information on how former applicants have done in graduate study and both their local norms (whether actually computed or not) and the examination's national test norms. Clinicians use scores from well-established measures such as the MMPI in much the same way. Experienced test users can make highly skilled interpretations on the basis of the proper test information, especially if they are also able to gather the other information that they need to fine-tune their judgments. Just translating a test and using the same scoring algorithm may not lead to meaningful scores, even when the understanding of specific scores is well-established in the original language, nation, or culture. Cultural and other differences between the original and target populations as well as language differences across the two forms of the assessment device may render the use of the same scores to denote identical meaning across test forms and cultures substantially less meaningful (Lu, 1967, as quoted by Cheung, 1985, p. 131).
  8. 8. References: American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1985). Standards for educational and psychological testing. Washington,DC: American Psychological Association. American Psychological Association (Producer). (1974-1992). Psychological abstracts. (From PsycLIT, Version 3.11 [CD-ROM]. Washington, DC: [Producer]. Boston: Silver Platter International [Distributor].) Angoff, W. H. (1971). Scales, norms and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education. Ben-Porath, Y. S. (1990). Cross-cultural assessment of personality: The case for replicatory factor analysis. In J. N. Butcher & C. D. Spielberger (Eds.), Advances in personality assessment (Vol. 8, pp. 1-26). Hillsdale, NJ: Erlbaum. Butcher, J. N. (1985). Current developments in MMPI use: An international perspective. In J. N. Butcher & C. D. Spielberger (Eds.), Advances in personality assessment (Vol. 4, pp. 83- 94). Hillsdale, NJ: Erlbaum. Butcher, J. N. Cheung, F. M. (1985). Cross-cultural considerations for the translation and adaptation of the Chinese MMPI in Hong Kong. In J. N. Butcher & C. D. Spielberger (Eds.), Advances in personality assessment (Vol. 4, pp. 131-158). Hillsdale, NJ: Erlbaum. Geisinger, K. F. (1994). Cross-cultural normative assessment: Translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychological assessment, 6(4), 304. International Test Commission, European Association of Psychological Assessment, International Association of Applied Psychology, International Association of Cross- Cultural Psychology, International Association for the Evaluation of Educational Achievement, International Language Testing Association, & International Union of
  9. 9. Psychological Science. (1993). Standards for adapting instruments and establishing score equivalence. Manuscript in preparation. Mininel, V. A., Felli, V. E. A., Loisel, P., & Marziale, M. H. P. (2012). Cross-cultural adaptation of the Work Disability Diagnosis Interview (WoDDI) for the Brazilian context. Revista Latino-Americana de Enfermagem, 20, 27-34. Pennock-Roman, M. (1990). Test validity and language background: A study of Hispanic- American students at six universities. New York: College Entrance Examination Board. Pennock-Roman, M. (1992). Interpreting test performance in selective admissions for Hispanic students. In K. F. Geisinger (Ed.), Psychological testing of Hispanics (pp. 99-135). Washington, DC: American Psychological Association.

×