Cross cultural surveys and linkert scales
•Download as PPTX, PDF•
1 like•279 views
Are you a researcher getting vastly different responses on client satisfaction scores? Take a look at this deck I put together for a dilemma we were attacking for our larger international audiences.
Recommended
More Related Content
Viewers also liked
Viewers also liked (8)
Similar to Cross cultural surveys and linkert scales
Similar to Cross cultural surveys and linkert scales (20)
Recently uploaded
Recently uploaded (20)
Cross cultural surveys and linkert scales
- 1. Cultural Biases in Surveys Sarah Miller March 12, 2013
- 2. Our Questions Japan - Are they always going to say “yes”? Brazil - Are Brazilians just happy? France - Is anything good enough for the French? Germany - Are the Germans just not fun? Why are your survey responses for Germany & France so different from Brazil & Japan? Is your survey adapted for culture?
- 3. How are they responding currently? Japan (-) Brazil (+) France (negative) Germany (negative)
- 4. Maybe they have one or more of these biases Acquiescence Response Style (ARS) Where is it a problem? Agree/Disagree questions Middle Response Style (MRS) Extreme Response Style (ERS) Response style bias: is there a tendency to respond to survey questions in a particular way, regardless of content?
- 5. ARS Acquiescence Response Style (ARS) Where is it Prevalent? Cultures where being polite is important Can we pinpoint the issue? Agree/Disagree questions are tough Some cultures interpret these questions as a double negatives (agreeing with something negative is doubly negative) They may not fully disagree or agree
- 6. MRS Middle Response Style (MRS) Where are MRS prevalent? Cultures where extremes are discouraged: Asian Cultures Can we pinpoint the issue? Confucious: No one answer is absolute Collectivism: Avoid making others look bad
- 7. ERS Extreme Response Styles Where are ERS prevalent? In cultures that value individualism, are masculine, and have a low tolerance for ambiguity Can we pinpoint the issue? Individualistic, masculine cultures put a value on assertiveness and decisiveness
- 8. Geert wants to remind us… Power Distance (PDI) Individualism (IDV) Uncertainty Avoidance (UA) Masculinity (MAS) Long Term Orientation (LTO) Response biases are related to all of these orientations
- 9. Brazil: High Acquiescence Highest acquiescence cultures… and may have a bit of SDR Why? High UA & Low IDV Collectivism (low IDV) Expected behaviors More collectivist than Japan! Aversion to confrontation Politeness
- 10. Japan is a Middle Response Culture Why? Collectivism Personal opinions are not strongly voiced Value harmony Dialectical Thinkers No one, absolute truth Long Term Orientation LTO societies believe that truth depends on the situation
- 11. Germans are well balanced, but can be a little negative Why? DRS or “disacquiescence” is associated with low power distance and high individualism. Germany is be a prime example of this type of culture. High masculinity and UA are also associated negatively to acquiescence. PDI: 35 IDV: 67 MAS: 66 UA: 65
- 12. The French are Extremists! The French are the 3rd most extreme culture in Europe. Why? Rules (UA) Societies higher in UA have many rules and have little tolerance for ambiguity. Direct Communication (IDV) Individualistic societies are characterized by self- expression, independence, and personal opinions. Individual contributions expected, speaking up appreciated – disagreement is normal. Individualistic societies have absolute truths UA: 86 IDV: 71 "You Can't Judge a Book by its Cover", Arun Joshi, Sagar Tamang, Himanshu Vashishtha, Research World, January 2009 International Marketing Research By C. Samuel Craig, Susan P. Douglas
- 13. Solutions Linkert 5 point scales are not the best solution across these response styles & cultures
- 14. How to balance acquiescent cultures What’s the Solution? Give various levels of agreement and disagreement Reverse order points of scale Employ negatively worded statements vs positive Or don’t use obvious highs/lows or negatives/positives Use visual stimuli/pictures vs words or numbers Test for acquiescence using a behavioral variable Joshi-Tamang-Vashishtha Scale Acquiescence is closely related to Social Desireability – lower class people will give responses that reflects the values of the interviewer in face to face interviews
- 15. Solutions for MRS Cultures What’s the Solution? Kill the middle response! Use a multi-item scale
- 16. Solutions for the Extreme Response Styles What’s the solution?
- 17. Survey Development Survey Type Item Generation Survey Languages
- 18. Survey Type Not all surveys are transferable Scale development is important Measurement scales need to be investigated
- 19. Item Generation Does the survey rely on commonalities or distinctions? Emic or Etic? Applicable universal concepts, values, truths Frame of reference - not just language - should be conceptually equivalent Suggestion Qualitative data (interviews) can be helpful, especially if taking surveys from multiple countries at once
- 20. The Role of Language English language by default elicits middle responses Opposites in one language may not be opposite in another language Double negatives! Intensity levels or words could be misinterpreted Direct translation isn’t always the key Scale anchors may not translate well Harzing, 2006
- 21. The Role of Language Language impacts surveys! Cultural accommodation Symbolic effect Concepts and terms Interpretation – culture specific connotations Researcher needs cultural knowledge as well as language knowledge Use clear, simple, and familiar wording Use natives in country where surveying Careful translation + pilot testing with native speakers
- 22. Other Solutions Mixture of Positive & Negative Statements Mitigates acquiescence and disaquiescence Linkert scales, use more than 5! Reduces response and language bias Scale anchors that have mutually exclusive constructs rather than agreement level also reduce biases* Do not compare loyalty scores across countries, instead compare loyalty scores across companies in the same country! Less complex than statistical adjustments and probably more accurate
- 23. NOT Good Solutions Some organizations try to eliminate the influence of response bias by calculating a cross-cultural adjustment factor for each country or region in which they do business. The adjustment factor is calculated as the ratio of the average score for a comparison country over the average score for a base country, which is typically the organization’s largest, most important market. In practice, however, this method is ill-advised unless the adjustment procedure truly isolates opinion differences from differences due to cultural response bias
- 24. Remember The researcher needs to take culture into consideration in the overall design, language, and item generation of the survey. Benchmarks should be within the same country More points on the Linkert scale Cross-country comparison helps to identify survey issues Native/local speakers can help with design
Editor's Notes
- Response Styles (ARS) acquiescence The tendency to agree with an item regardless of the content (DRS) disacquiescence The tendency to disagree with an item regardless of the content (ERS) Extreme Response Styles The tendency to use the extreme response categories on ratings scales (MRS) Middle Response Styles The tendency to use middle response categories on ratings scales (AMBRS) – Ambivalent Response Styles (SD) – Social Desirability
- Thurstone scaling?
- Japan and Brazil are collectivist societies, making the group more important than the individual. These societies tend to not speak directly nor strongly voice personal opinions. Individualistic societies (USA, Germany, France) put an emphasis on the individual and encourage voicing and arguing opinions. Underlying reasons for acquiescence bias 1. Collectivity One critical factor for acquiescence bias within a society is its level of collectivity. In cultures that are marked by a high degree of collectivism, individuals feel it as important to consider what people might think and say about their actions, and this creates a bias. According to Hofstede, collectivist countries tend to be characterized by a preference for harmony and an avoidance of confrontations, leading to more conformity in behavior where opinions are predetermined by the ingroup . This tendency is reinforced by the fact that most collectivist countries are characterized by an indirect communication style, where the expression of strong opinions is avoided. Confucian teachings (predominant in East Asian countries) reinforce this communicationstyle by advising followers to keep themselves from extremes ( 2. Politeness Another factor is politeness, the level and nature of hospitality displayed in a society can have an impact on acquiescence bias. If a face-to-face interview takes place at the respondent’s place of residence, the respondent will treat the interviewer as a guest and exhibit behavior that befits a good host. 3. Positivity In a culture that preaches being positive, ‘negativity avoidance’ tends to create acquiescence bias meaning that whilst a respondent might well have a negative feeling or reaction, they would tend to avoid giving a negative response. Other factors such as economic prosperity, consumer confidence and a high degree of materialistic indulgence tend to add to acquiescence bias (Joshi, 2009). It is important to note that all of the countries being compared are among the top 10 economic performers in terms of GDP & PPP (CIA World Factbook, 2012).
- Middle Response Style (MRS): Japan The reasoning behind Japan’s MRS – and slightly acquiescent style – is due to at least three major factors. These three major factors in Japanese and East Asian cultures are influenced by Confucian teachings. Japan is collectivist (low IDV) The collectivist mindset may cause Japanese participants to choose the middle response to avoid any upset. Confucian teachings reinforce this communication style by advising followers to keep themselves from extremes (Hertzing, 2012). Japan is a country of dialectical thinkers: No one, absolute truth East Asian countries are characterized by dialectical thinking - or arguing against oneself, seeing multiple solutions. Western notion of ‘truth’ has different connotations in Confucian Philosophy (Harzing, 2012). Japanese are long-term oriented (LTO) In societies with a long-term orientation, people believe that truth depends very much on situation, context and time (Hofstede). Dialectical thinking also falls in line with Confucian teachings – Dialectical thinking: Cultures differ indialectical thinking, that is, the tolerance for holding apparently contradictory beliefs (Peng & Nisbett, 1999). In particular, dialectical thinking appears to be more common among East Asians than North Americans. For example, Peng and Nisbett (1999) found that Chinese often endorsed both of two arguments that Americans viewed as incompatible. Dialectical thinking is also evident in the way that East Asians think of themselves. For example, Choi and Choi (2002) showed that, compared to Americans, Koreans displayed self-beliefs that were less consistent (e.g., I’m outgoing but somewhat shy). Spencer-Rodgers, Peng, Wang, and Hou (2004) found that, compared to Americans, Chinese self-evaluations are more ambivalent in the sense of incorporating both positive and negative self-evaluations.
- Joshi-Tamang-Vashishtha Scale
- Survey Type Scales Many scales are available, but may not be transferable to a different cultural context. The use of multinational research teams whose members are familiar with the respective local cultures may help to overcome problems related to adapting measurement scales. Key to a meaningful modification of existing measurement scales is a sound process of scale development.
- Item Generation Construct Equivalence Choose whether or not your survey construct relies upon the commonalities between the benchmark culture and the country to be studied (etic) or the intrinsic distinctions between the two cultures (emic). Emic/Etic: A construct can only be meaningfully measured across cultures if it is based on universally applicable concepts in these cultures, that is, the construct is conceptually equivalent. In this regard, whereas questionnaire translation is necessary to ‘clarify’ construct elements in the local language and frame of reference it is an insufficient condition for establishing construct equivalence (Harzing, 2012). Triangulation: The feasibility of modifying existing scales to accommodate for cultural specificities is therefore limited if data from a larger amount of countries are to be compared. In that case, the collection of qualitative data, for example through interviews, can compensate for the inherent limitations of survey data. Triangulation can thus increase the robustness of the data gathered.
- English vs Native Language English-language questionnaires are shown to elicit a higher level of middle responses, while questionnaires in a respondent’s native language result in more extreme response styles. Finally, English language competence is positively related to extreme response styles and negative related to middle response styles. An important explanation for response style differences across languages might be differential interpretation of equivalent scale anchors in different languages. Even though scale anchors might translate into appropriate local equivalents, the intensity associated with these equivalents might be different from the original language. (Harzing, 2006). Most studies that show response bias used Likert scales with ordered scale anchors, e.g. “strongly disagree” to “strongly agree” or “of little or no importance” to “extremely important”. These anchors might be vulnerable to acquiescence bias as respondents are keen to agree, whether this is caused by high levels of power distance, collectivism, uncertainty avoidance or extraversion. Our detailed analysis showed that scale anchors referring to the level of importance are even more problematic in this respect than scale anchors referring to the level of agreement. A related problem in this respect – discussed above with special reference to Japan – is that scale anchors are often difficult to translate and that translations might not result in metric equivalence. An alternative would be to use scale anchors as part of the question and let them reflect opposites rather than level of agreement. This would make the “right answer” less obvious and would also force respondents to carefully consider each question as most scale anchors would be different. Of course this technique would increase the level of cognitive involvement required and might lead to lower response rates. In addition, careful translation and pilot testing would become even more crucial as a respondent’s interpretation of the questions would be framed by single words, whereby words that are seen as opposites in some countries might not be opposites in other countries. However, if translation problems can be solved, responses might be more meaningful (Harzing, 2006).
- Survey Languages If distributing a survey in English, it is important to consider language proficiencies. In some cases, English may in fact be adequate. This would be the case for very specific populations. Research shows that English-language questionnaires may create a bias Translation is crucial Survey translation & interpretation is crucial. Language of the questionnaire can impact attitudes and behaviors (cultural accommodation). The translation and interpretation into a local language – even when the respondent has a high level of English - has a symbolic effect on the participant. The fact that the researcher took the time and effort to make the survey as easy as possible for the participant has an impact. Concepts and terms differ across cultures. Asian countries regard feedback as personal, indirect, and informal procedure. The US & UK regard feedback as an open, direct, and formalized procedure (Hofstede, 1998) Interpretation is necessary as many concepts and terms entail culture-specific connotations, their mere direct translation is unlikely to transport the intended meaning A meaningful translation of the original version of the questionnaire requires a researcher not only to ensure overall conceptual equivalence but also to consider vocabulary, idiomatic and syntactical equivalence Suggestions: It is suggested to use simple sentence structures as well as clear and familiar wording as much as possible to facilitate translation. In addition, by adding redundancy and necessary context for difficult phrases, the researcher is able to clarify the intended meaning (Harzing, 2012). Pilot testing is particularly important in international research. One way to achieve this is to use a committee to review the survey to see if the interpretations of the survey are the same. In terms of the choice of survey type, it is advisable to obtain help from native speakers in designing the survey, ask locals to check and test the survey, and combine both paper-andpencil and online surveys to increase response rates. To generate survey items, it is important to fist decide whether the research project is emic or etic in nature. Whereasthe former case requires the adaptation of existing and development of new items, in the latter case this may be unfeasible when interested in cross-cultural comparability. Instead, researchers should collect additional qualitative data to triangulate the study results. Finally, several translation techniques exist to adapt the survey to the local language of the countries it is diffused. Simple sentence structures should be used to facilitate translation, and additional clarifications provided to better convey the intended meaning of certain questions. It is also important to pilot test the survey in each country (Harzing, 2012). Other solutions all relate to initial questionnaire design. A use of a mixture of positive and negative statements will mitigate both acquiescence and disacquiescence. Likert scales with a larger number of scale points and the use of ranking have also been shown to reduce 31 | Page both response and language bias (Harzing et al., 2009) as have scale anchors that refer to mutually exclusive constructs, rather than to level of agreement (Harzing et al., 2012).
- These are not solutions: For example, researchers will typically assess customer opinions across a minimum of three areas unrelated to the area of actual interest, by adding additional questions to the survey. Surveys are fielded to an international panel of consumers and the results are used to construct structural equations or constrained factor analyses which help to isolate the cultural response bias from differences in true opinion (Brunso & Guerent,1995; Weijters, et. al., 2004; King, et. al., 2004; Keillor et. al., 2001). Some organizations try to eliminate the influence of response bias by calculating a cross-cultural adjustment factor for each country or region in which they do business. The adjustment factor is calculated as the ratio of the average score for a comparison country over the average score for a base country, which is typically the organization’s largest, most important market. In practice, however, this method is ill-advised unless the adjustment procedure truly isolates opinion differences from differences due to cultural response bias. If this is not the case (and short of the statistical modeling described above, it is difficult to do so), the adjustment procedure is likely to eliminate true differences in opinion across countries. the exact information the customer survey is intended to measure. Further, this approach does not permit accurate statistical tests of observed differences across countries, making it difficult to assess what constitutes a truly .significant. discrepancy in scores (Scholderer et. al., 2005).