Evaluation Studies of Cultural Awareness Training


Published on

Evaluation Studies of Cross-Cultural Training Program: A Review of the Literature From 1988 to 2000. An Executive Summary.

Published in: Business, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Evaluation Studies of Cultural Awareness Training

  1. 1. Hangzhou, March 2011 Prof. Dr. Hora Tjitra & Mu Tong, Zhejiang University Evaluation  Studies   of  Cross-­cultural  Training A Review of the Literature From 1988 to 2000
  2. 2. Evaluation  Studies    of  Cross-­‐Cultural  Training Evaluation Studies on Cultural Awareness Training Black and Mendenhall (1990) Deshpande and Viswesvaran (1992) Bhagat and Prien (1996) Kealey and Protheroe (1996) Bhawuk and Brislin (2000)
  3. 3. Evaluation  Studies    of  Cross-­‐Cultural  Training 3 14  years  in  Germany 7  years  in  China Born  and  grew  up   in  Indonesia Prof.Dr.Hora Tjitra - Cross-cultural and Business Psychology Dipl.-Psych.,Technical University of Braunschweig Organizational Psychology and Human Resource Management Dr.Phil.,University of Regensburg Intercultural Psychology and Strategic Management Executive Education,INSEAD HR Management in Asia
  4. 4. Evaluation  Studies    of  Cross-­‐Cultural  Training Key Benefits of Evaluation Learning Value Chain and the Benefits of its Measurements LevelLevel Key Questions 0 1 2 3 4 5 Input and Indicators What is the number of participants,hours, and programs and what are the costs? Reaction and Planned Action Was the learning relevant,important, useful,and helpful to participants in the job environment? Did the participants plan to use the content in the program? Learning and Confidence Did participants increase or enhance knowledge,skills,or perceptions and have confidence to use them? Application & Implementation What did the participants do differently in the job context? Was the program implemented effectively? What changes were made on the job? Impact and Consequences What are the consequences of the application in terms of output,quality, cost,time,and satisfaction? ROI Did the monetary benefits of the learning program exceed the investment in the program? Sources:Elkele & Philips (2007).The Chief Learning Officer. Satisfy client needs Justifying budgets Improve program designs and processes Enhance the transfer of learning Enhance the respect and credibility of LTD staff Increase support from line managers Strengthen relationship with key executive and administrators Set priorities for learning and development 4
  5. 5. Evaluation  Studies    of  Cross-­‐Cultural  Training Key Benefits of Measurement and Evaluation • Sa#sfy  client  needs • Jus#fying  budgets • Improve  program  designs  and   processes • Enhance  the  transfer  of  learning • Eliminate  of  expand • Enhance  the  respect  and  credibility   of  LTD  staff • Increase  support  from  managers • Strengthen  rela#onship  with  key   execu#ve  and  administrators • Set  priori#es  for  learning  and   development 208 The Chief Learning Officer The ROI Process Evaluation Planning Data Collection Collect Data During Solution Implementation LEVEL 1: REACTION, SATISFACTION, AND PLANNED ACTIONS LEVEL 2: LEARNING Collect Data After Solution Implementation LEVEL 3: APPLICATION/ IMPLEMENTATION LEVEL 4: BUSINESS IMPACT Develop Objectives of Solution(s) Develop Evaluation Plans and Baseline Data Convert Data to Monetary Value Calculate the Return On Investment Generate Impact Study Data Analysis Identify Intangible Measures LEVEL 5: ROI Reporting INTANGIBLE BENEFITS Isolate the Effects Tabulate Costs Of Solution Figure 8-4. ROI methodology. (Source: Phillips, 2003. Used with permission.) 5
  6. 6. Evaluation  Studies    of  Cross-­‐Cultural  Training Tools and methods in learning evaluation Collecting Data Isolating the Effects of Learning Converting Data to Monetary Values •Surveys •Questionnaires •Tests •On-the-job observation •Interviews •Focus groups •Action plan and program assignments •Performance contracts •Business performance monitoring •Control group •Pre-post measurements •Trend lines and forecasting •Participants or managers •Other experts •Output data is converted to profit contribution or cost savings •The cot quality is calculated •Wages and benefits for the used of time value •Historical cost and current used •Opportunity cost •External database •Participant estimation from different groups 6
  7. 7. Evaluation  Studies    of  Cross-­‐Cultural  Training 7 When to Use the Training: International Assignment • Developing expatriate-profiles and conducting selection • Preparation of the expatriates and their families (e.g. cultural awareness training, etc.) • Conceptualization of an escorting system during the assignment (e.g. coaching, mentoring, etc.) • Return planing and reintegration seminar • Internal marketing to motivate employee for international assignment Marketing Selection PreparationSupporting Reintegration International   Assignment Different  types  of  cultural  awareness  training  is  helpful  for  each  stage  of  the  circle 10%  -­‐  50%  of  expatriates  returning  early  from  their  assignment.   Expatriates’  difficulAes  are  costly  for  MNCs,  ranging  from  $250,000  to  $1  million.                                                                                                                                                                                                                                                            (Eschbach  et  al.,  2001)
  8. 8. Evaluation  Studies    of  Cross-­‐Cultural  Training Generic Design of Cross Cultural Awareness Training 8 Lunch Dinner Break Break Day 3 … on the Way to Cross-Cultural Synergy Personal Charter & Learning Plan Day 1 Critical Situations in Working Places My understanding of others: Self and other perception Culture & Management: Working in International Environments Cultural Encounter Situations Learning in Action Short Lecture & Discussion Joint Dinner Case Study: Cross-cultural Leadership Day 2 Communication Across Cultures Role Play: “First Meeting” Case and Role Play: Conflict Management and Team Building
  9. 9. Evaluation  Studies    of  Cross-­‐Cultural  Training How Effective is a Cross-Cultural Awareness Training in Developing Intercultural Competence? 9
  10. 10. Evaluation  Studies    of  Cross-­‐Cultural  Training Summary of the Evaluation Studies on Awareness Training • Some evaluation confirms the effect of CCT on trainees’ cross-cultural skill, adjustment, and performance. • Others have doubts regarding the quality of these studies whether CCT effectiveness has been demonstrated. • Don’t establish a “link” between training and the measured effect on a dependent variable. • The effectiveness of CCT in these studies was regarded as too positive in previous reviews. • In general, it remains unclear which method of CCT is overall most effective and which methods are most effective in specific contingencies. • The result for the same studies does not always correspond in different reviews due to the different perspectives taken by the reviewers. Littrell & Salas, 2005
  11. 11. Evaluation  Studies    of  Cross-­‐Cultural  Training  English language vs. Multiple languages: Eglish,French,German, Italian,and Spanish.  Specific domains vs. Wide variety of disciplines: Management, military studies, human resource management, cross-cultural psychology, intercultural relation, immigrant studies, education, anthropology, political science and diplomacy. Domain of the Evaluation Study
  12. 12. Evaluation  Studies    of  Cross-­‐Cultural  Training Findings Methods ✓ Lectures, culture assimilators and class discussions were the most often used. ✓ Training was composed of three or more methods . ✓ U.S. trainees were evaluated most. ✓ Over half of the treatment groups were students. ✓ Lack of manager subjects. Participants
  13. 13. Evaluation  Studies    of  Cross-­‐Cultural  Training Training Methods Used in the Evaluation Studies
  14. 14. Evaluation  Studies    of  Cross-­‐Cultural  Training Findings Focus ✓ Most training focus on a specific culture. ✓ Others were culture general or a combination. ✓ Japanese, U.S. and Canadian cultures were the most common focus. Timing ✓ The training took place either prior to, during, or independently of a sojourn in another country. ✓ The duration of the training varied from1/2 to 50 hours over a period of time ranging from 1 day to 8 months.
  15. 15. Evaluation  Studies    of  Cross-­‐Cultural  Training Trainee satisfaction Academic achievement, contribution to company goal. Perceived well-being, health and safety concerns, adjusted to a foreign culture, perceived control of the environment. The ability to make isomorphic attributions awareness of cultural differences Cultural interest, positive attitude toward members of other cultures, trainee ethnocentrism. Problem-solving ability, the ability to deal with cross-culture misunderstandings, display of cultural sensitivity Expected Results of Awareness Training Knowledge Behavior Attitude Adjustment Performance Satisfaction
  16. 16. Evaluation  Studies    of  Cross-­‐Cultural  Training Cross-Cultural Training Effectiveness
  17. 17. Evaluation  Studies    of  Cross-­‐Cultural  Training Some Critical Issues of the Current Evaluation Studies • Control Groups • Pre- and Post-Training Test • Randomly Assign • Long-term vs. Short Term Effects • Different Measurement Methods • Placebo and Hawthorne Effects
  18. 18. Evaluation  Studies    of  Cross-­‐Cultural  Training 18 International Project References in the Research Areas of Culture Diversity,Talent Development,and Strategic Change 14  years  in  Germany Born  and  grew  up   in  Indonesia 8  years  in  China • Building Global Competence for Asian Leaders • Applying Social Medias (Web 2.0) in Learning & Development. • International Employability:Development of Intercultural Competence of German and Chinese Young Professionals. • Cross-Cultural Learning Behavior: Effectiveness of the Western Technology Transfer and Learning Approaches in China. • Comparative Studies of Chinese-Indonesian Intercultural Competence and Sensitivity. • Dynamic Decision Making in Chinese and Multinational Teams. • Intercultural Perspectives of International Post-Merger Integration in Europe. • Intercultural Synergy in Professional Team. • Complex Problem Solving in Small Groups. • ...
  19. 19. Evaluation  Studies    of  Cross-­‐Cultural  Training Conclusions and Implications ✦ Quantitative and Qualitative ✦ Time Span ✦ Wintry Doctrine ✦ Low Motivation ✦ Moderators ✦ The Paucity of theory ✦ Cooperation and Resistance
  20. 20. Thank  You Contact us via … Mail: hora_t@mac.com Follow: twitter@htjitra Website: http://horatjitra.com Zhejiang  University,  Hangzhou  (China) Based on: LISA N. LITTRELL, EDUARDO SALAS: A Review of Cross-Cultural Training: Best Practices, Guidelines, and Research Needs. Human Resource Development Review, 4(3), 2005. p305-334