Evaluation Studies of Cross-Cultural Training Program: A Review of the Literature From 1988 to 2000. An Executive Summary.

1. Hangzhou, March 2011
Prof. Dr. Hora Tjitra & Mu Tong, Zhejiang University
Evaluation
Studies
of
Cross-­cultural
Training
A Review of the Literature From 1988 to 2000

2. Evaluation
Studies
of
Cross-­‐Cultural
Training
Evaluation Studies on Cultural Awareness Training
Black and Mendenhall
(1990) Deshpande and Viswesvaran
(1992)
Bhagat and Prien
(1996)
Kealey and Protheroe
(1996)
Bhawuk and Brislin
(2000)

3. Evaluation
Studies
of
Cross-­‐Cultural
Training
3
14
years
in
Germany
7
years
in
China
Born
and
grew
up
in
Indonesia
Prof.Dr.Hora Tjitra - Cross-cultural and Business Psychology
Dipl.-Psych.,Technical University of Braunschweig
Organizational Psychology and Human Resource Management
Dr.Phil.,University of Regensburg
Intercultural Psychology and Strategic Management
Executive Education,INSEAD
HR Management in Asia

4. Evaluation
Studies
of
Cross-­‐Cultural
Training
Key Benefits of Evaluation
Learning Value Chain and the Benefits of its Measurements
LevelLevel Key Questions
0
1
2
3
4
5
Input and
Indicators
What is the number of participants,hours,
and programs and what are the costs?
Reaction and
Planned Action
Was the learning relevant,important,
useful,and helpful to participants in the job
environment? Did the participants plan to
use the content in the program?
Learning and
Confidence
Did participants increase or enhance
knowledge,skills,or perceptions and
have confidence to use them?
Application &
Implementation
What did the participants do differently in
the job context? Was the program
implemented effectively? What changes
were made on the job?
Impact and
Consequences
What are the consequences of the
application in terms of output,quality,
cost,time,and satisfaction?
ROI
Did the monetary benefits of the learning
program exceed the investment in the
program?
Sources:Elkele & Philips (2007).The Chief Learning Officer.
Satisfy client needs
Justifying budgets
Improve program designs
and processes
Enhance the transfer of learning
Enhance the respect and credibility of LTD
staff
Increase support from line managers
Strengthen relationship with key executive
and administrators
Set priorities for learning and development
4

5. Evaluation
Studies
of
Cross-­‐Cultural
Training
Key Benefits of Measurement and Evaluation
• Sa#sfy
client
needs
• Jus#fying
budgets
• Improve
program
designs
and
processes
• Enhance
the
transfer
of
learning
• Eliminate
of
expand
• Enhance
the
respect
and
credibility
of
LTD
staff
• Increase
support
from
managers
• Strengthen
rela#onship
with
key
execu#ve
and
administrators
• Set
priori#es
for
learning
and
development
208 The Chief Learning Officer
The ROI Process
Evaluation
Planning
Data Collection
Collect Data
During Solution
Implementation
LEVEL 1:
REACTION,
SATISFACTION, AND
PLANNED ACTIONS
LEVEL 2:
LEARNING
Collect Data
After Solution
Implementation
LEVEL 3:
APPLICATION/
IMPLEMENTATION
LEVEL 4:
BUSINESS IMPACT
Develop
Objectives
of
Solution(s)
Develop
Evaluation
Plans and
Baseline Data
Convert Data
to Monetary
Value
Calculate
the Return On
Investment
Generate
Impact
Study
Data Analysis
Identify
Intangible
Measures
LEVEL 5: ROI
Reporting
INTANGIBLE BENEFITS
Isolate the
Effects
Tabulate
Costs
Of Solution
Figure 8-4. ROI methodology. (Source: Phillips, 2003.
Used with permission.)
5

6. Evaluation
Studies
of
Cross-­‐Cultural
Training
Tools and methods in learning evaluation
Collecting Data Isolating the Effects of Learning Converting Data to Monetary Values
•Surveys
•Questionnaires
•Tests
•On-the-job observation
•Interviews
•Focus groups
•Action plan and program assignments
•Performance contracts
•Business performance monitoring
•Control group
•Pre-post measurements
•Trend lines and forecasting
•Participants or managers
•Other experts
•Output data is converted to profit
contribution or cost savings
•The cot quality is calculated
•Wages and benefits for the used of
time value
•Historical cost and current used
•Opportunity cost
•External database
•Participant estimation from different
groups
6

7. Evaluation
Studies
of
Cross-­‐Cultural
Training
7
When to Use the Training: International Assignment
• Developing expatriate-profiles
and conducting selection
• Preparation of the expatriates and
their families (e.g. cultural
awareness training, etc.)
• Conceptualization of an escorting
system during the assignment
(e.g. coaching, mentoring, etc.)
• Return planing and
reintegration seminar
• Internal marketing to motivate
employee for international assignment
Marketing
Selection
PreparationSupporting
Reintegration
International
Assignment
Different
types
of
cultural
awareness
training
is
helpful
for
each
stage
of
the
circle
10%
-­‐
50%
of
expatriates
returning
early
from
their
assignment.
Expatriates’
difficulAes
are
costly
for
MNCs,
ranging
from
$250,000
to
$1
million.
(Eschbach
et
al.,
2001)

8. Evaluation
Studies
of
Cross-­‐Cultural
Training
Generic Design of Cross Cultural Awareness Training
8
Lunch
Dinner
Break
Break
Day 3
… on the Way
to Cross-Cultural Synergy
Personal Charter
& Learning Plan
Day 1
Critical Situations
in Working Places
My understanding of others:
Self and other perception
Culture & Management:
Working in International
Environments
Cultural Encounter
Situations
Learning in Action
Short Lecture & Discussion
Joint Dinner
Case Study:
Cross-cultural Leadership
Day 2
Communication
Across Cultures
Role Play:
“First Meeting”
Case and Role Play:
Conflict Management and
Team Building

9. Evaluation
Studies
of
Cross-­‐Cultural
Training
How Effective is a Cross-Cultural Awareness Training
in Developing Intercultural Competence?
9

10. Evaluation
Studies
of
Cross-­‐Cultural
Training
Summary of the Evaluation Studies on Awareness Training
• Some evaluation confirms the effect of CCT on trainees’
cross-cultural skill, adjustment, and performance.
• Others have doubts regarding the quality of these studies
whether CCT effectiveness has been demonstrated.
• Don’t establish a “link” between training and the measured
effect on a dependent variable.
• The effectiveness of CCT in these studies was regarded as
too positive in previous reviews.
• In general, it remains unclear which method of CCT is overall
most effective and which methods are most effective in
specific contingencies.
• The result for the same studies does not always correspond
in different reviews due to the different perspectives taken by
the reviewers.
Littrell & Salas, 2005

11. Evaluation
Studies
of
Cross-­‐Cultural
Training
 English language vs. Multiple languages:
Eglish,French,German,
Italian,and Spanish.
 Specific domains vs. Wide variety of disciplines:
Management, military studies,
human resource management,
cross-cultural psychology,
intercultural relation,
immigrant studies,
education, anthropology,
political science and diplomacy.
Domain of the Evaluation Study

12. Evaluation
Studies
of
Cross-­‐Cultural
Training
Findings
Methods
✓ Lectures, culture assimilators and class
discussions were the most often used.
✓ Training was composed of three or
more methods .
✓ U.S. trainees were evaluated most.
✓ Over half of the treatment groups
were students.
✓ Lack of manager subjects.
Participants

13. Evaluation
Studies
of
Cross-­‐Cultural
Training
Training Methods Used in the Evaluation Studies

14. Evaluation
Studies
of
Cross-­‐Cultural
Training
Findings
Focus
✓ Most training focus on a specific culture.
✓ Others were culture general or a
combination.
✓ Japanese, U.S. and Canadian cultures
were the most common focus.
Timing
✓ The training took place either prior to,
during, or independently of a sojourn
in another country.
✓ The duration of the training varied
from1/2 to 50 hours over a period of
time ranging from 1 day to 8 months.

15. Evaluation
Studies
of
Cross-­‐Cultural
Training
Trainee satisfaction
Academic achievement,
contribution to company goal.
Perceived well-being, health and safety concerns, adjusted to a
foreign culture, perceived control of the environment.
The ability to make isomorphic attributions
awareness of cultural differences
Cultural interest, positive attitude toward members of other
cultures, trainee ethnocentrism.
Problem-solving ability, the ability to deal with cross-culture
misunderstandings, display of cultural sensitivity
Expected Results of Awareness Training
Knowledge
Behavior
Attitude
Adjustment
Performance
Satisfaction

16. Evaluation
Studies
of
Cross-­‐Cultural
Training
Cross-Cultural Training Effectiveness

17. Evaluation
Studies
of
Cross-­‐Cultural
Training
Some Critical Issues of the Current Evaluation Studies
• Control Groups
• Pre- and Post-Training Test
• Randomly Assign
• Long-term vs. Short Term Effects
• Different Measurement Methods
• Placebo and Hawthorne Effects

18. Evaluation
Studies
of
Cross-­‐Cultural
Training
18
International Project References in the Research Areas of
Culture Diversity,Talent Development,and Strategic Change
14
years
in
Germany
Born
and
grew
up
in
Indonesia
8
years
in
China
• Building Global Competence for Asian Leaders
• Applying Social Medias (Web 2.0) in Learning & Development.
• International Employability:Development of Intercultural
Competence of German and Chinese Young Professionals.
• Cross-Cultural Learning Behavior:
Effectiveness of the Western Technology Transfer and Learning
Approaches in China.
• Comparative Studies of Chinese-Indonesian Intercultural
Competence and Sensitivity.
• Dynamic Decision Making in Chinese and Multinational Teams.
• Intercultural Perspectives of International Post-Merger
Integration in Europe.
• Intercultural Synergy in Professional Team.
• Complex Problem Solving in Small Groups.
• ...

19. Evaluation
Studies
of
Cross-­‐Cultural
Training
Conclusions and Implications
✦ Quantitative and Qualitative
✦ Time Span
✦ Wintry Doctrine
✦ Low Motivation
✦ Moderators
✦ The Paucity of theory
✦ Cooperation and Resistance

20. Thank
You
Contact us via …
Mail: hora_t@mac.com
Follow: twitter@htjitra
Website: http://horatjitra.com
Zhejiang
University,
Hangzhou
(China)
Based on: LISA N. LITTRELL, EDUARDO SALAS:
A Review of Cross-Cultural Training: Best Practices, Guidelines, and Research Needs.
Human Resource Development Review, 4(3), 2005. p305-334