E-Valuation Gabriella Lakatos
The Emphasis of Cross Cultural Awareness in E-learning Testing and
Evaluation Process
Equity within e-valuation
What elements cause difficulties and require the changes of the evaluation standards?
Toward the high-quality e-educational services
From the start of the use of computers in classrooms, Info technology and Internet made
borders virtually in education around the world. Large amount of e-learning courses, online
education systems have been increasing year by year.
Students, teachers and education experts have to face on challenges of multi culture.
Teaching, testing and evaluation standards must follow the tendency, because of
international courses and e-learning now are available for everyone. Estimated amount: 50%
of all learning by educational institutions, business, and industry by the year 2010 will be
done at a distance (Lori B. Holcomb Frederick B. King 2004).
Planning to design the Euro Bridge English Club e-Learning System for adults in 2004, I was
personally faced to the diversity of communication. Making the project I learnt that the most
difficult process is to set up appropriate evaluation tests for international groups.
Some aspects of my collection highlighted to my personal experiences:
Equity within e-valuation
New questions have been raised about the relationship between the technology and student
culture. Although people choose courses by personal approach and E-learning could be
personal more or less, participants never meet their examiners, and the outcome, have to be
highly standardized. The testing and evaluation process must offer consequent, and equal
outcome for each student doesn’t matter who they are where they come from. (Jia
Frydenberg 2002). E-valuation process must be impersonal! The question is how to fit the
generalized standards and the highly personalized demands to the multicultural diversity
within e-valuation process?
Intended learning outcomes are reviewed regularly to ensure clarity, utility, and
appropriateness.”
Who participates?
Teachers and E-tutors have to consider some existing elements of evaluation which were
less emphasized before The Information Age.
The purposes have changed students come from different areas of 5 continents.
The language of these courses is English mostly, although participants often speak
English as a second language.
All they have different language skills and they have cultural diversity.
They have nonequivalent learning skills related to different educational systems.
Some are familiar with Anglo –Saxon testing process, some are not.
They have different ages that make different aptitudes and make differs within the
speed of performance
They have different social behavior which could also cause difficulties.
Their skills of problem solving are traditionally different in a “rule based” society or in a
“communication based” society.
1

The best evidence is World's Smallest Political Quiz used in more than 420 schools in the
United States and around the world (theadvocates.org, www.theadvocates.org/quiz.html)
Economic Issues
(Choose A if you agree, M for Maybe, D if you
A M D
disagree.)
End \"corporate welfare.\" No government handouts to
business
End government barriers to international free trade
Let people control their own retirement; privatize
Social Security
Replace government welfare with private charity
Cut taxes and government spending by 50% or more
Your answer could rather be predicted by your social aptitude than your knowledge.
What elements cause difficulties and require the changes of the evaluation standards?
- In spite of relatively high language skills of participants, they generally need more
time to understand the correlation of text in multi choice or gap filing tests.
- Long sentences or regionally used expressions could confuse the meaning of the
task.
- The limited intercultural knowledge causes handicaps of nonnative participants.
- Some solutions of applicants are social based.
Teachers need to be aware of what to measure exactly within the evaluation process:
the level of knowledge,
the level of language skills or
the level of cultural skills.
If a test maker missed to separate the facts and figures carefully, the price would be
the credibility of the online course and e-valuation.
Of course, the applicants also need a desirable level of intercultural knowledge,
although the test and evaluation process should have to respect more their limited
knowledge.
Be rather practical than theoretical
Designs and consequent results of evaluation projects tend to have a new approach of
teachers who perform measure systems internationally. I would rather have a practical than
theoretical approach of e-valuation, so I had personally experienced and performed some e-
learning courses as a student before I made the Euro Bridge English Club e-valuation
program as a designer. Recently I took part in international distance learning and in e-
learning courses within my post gradual studies and I often learnt that there is a gap between
native and nonnative applicants in effectiveness, efficiency, and equity.
I made a survey last year and I still found mostly generalized statements of e-learning
evaluation yet.
Although Evaluation or assessment is defined in many aspects in educational and
government policy I hardly found source of evaluation and testing. Educational researches
equally focus on satisfaction of students and the evaluation of e-learning course itself, but
they don’t emphasize the importance of cultural diversity within the feedback.
2

Fortunately I had experiences both in teaching internationally as a teacher, and in learning in
international groups as a student. My own e-learning studies underline that the lack of e-
learning projects is the lack of credible e-valuation. (Graham Attwell, Jenny Hughes 2002.
2003.)
Without having possibility to collect details of others participants, let me share some of my
experiences with you. The examples below belong to several types of evaluation process,
but all they have a mutual platform – the culture diversity.
Social and culture based problem solving:
Misuse of language exam
ESL English language exams often give evidence about how applicants could be confused
when examiners fall into the trap of false evaluation. Language exams required the most
sophisticated test assets. Only the qualities of communication skills must be measured not
the facts, which are the data medium of communication.
Oral exam:
Description of a picture:
Applicant spoke fluent English and gave detailed description about the old timber house. She
was under scored because she could not name the thatched roof, she called it reed roof.
Listening task:
The topic is based on an airport short conversation, which requires experiences of flight.
Those, who never flown could have handicaps.
Written exam:
I remember the surprise of the native English examiners in some English language exam in
Budapest in 2002. They couldn’t understand why applicants lied instead of telling some
accepted excuse in EURO Exam writing issue. Their task was to ask the head officer to let a
day work off. The “liars” were relatively less scored, because teachers thought that they
didn’t really catch the task. The truth is that there were periods of some ruled based social
tradition in Hungary until late twenties which taught people to lie for benefits in this situation.
Applicants had some instinct reaction telling a lie instead of obvious alibi. It was a perfect
solution from their point of view.
Multi choice questions:
English pre-exam course:
City Nicknames
Boston, Massachusetts, is known as
a) Tea Town
b) Beantown
c) Red Sox City
The question may be interesting, but totally irrelevant in a language test without embedded in
some reading exe.
3

Wrong test items occurred everywhere.
Test for marketing professionals designed by eMA:
Local Values in Content:
The range of these ad fees is usually:
1. 100 $-300 $
2. 5005 $ -7400 $
3. 10000 $-15000 $
4. More than 15000 $
Some service fees can be valid in USA that has difficulties for nonnative applicants.
Example of Test for CeM eMarketers:
ASP’s perform their function:
1 at the company’ site
2 at the company offices
3 at the MIS department
4 off site
Beware of abbreviations!
ASP is not an adequate abbreviation; even in IT technology have more meaning, 3 of them
can be fitted to this question properly at least.
There is ASP in different use in Europe! The best is to avoid this fact unless it has been
described in the Curriculum previously.
Worldwide “known” facts and figures in the test issues:
In a summer course in Cornwall I took part in a charity quiz event with my host family.
Participants were middle class intellectuals and business men. I couldn’t answer only one of
the questions. It shocked me. I realized that I never heard about people who are “world
famous” and about plays which are “well known all around the world”, and vice versa: they
didn’t even know English authors and scientists who are “well known” in Hungary.
Surprisingly they only knew Richard Attenborough but did not know
David Attenborough. Being well known is extremely designed by media. It occurred in the e-
learning evaluation tests too, if E-tutors have no reliable multicultural information.
One nation – many cultures
Some say that it is not enough important for those “big nations” like US or GB, some say,
that it is. I agree with those who say, yes. More evidences can be found of the growing
importance of multi cultural awareness in the late twenties, than some though. These
countries have been faced to this problem earlier than the Internet based community. In US
or in GB health and education system revealed the problem first. In USA the search of cross
cultural awareness was promoted by the American health insurance business and it was
authorized in the clinical practice.
Plurality in European Community
Cultural diversity is accepted on the stage of political declarations in EU. The general
standards of European Union (CEFR Common European Framework of Reference for
Languages, Lifelong Learning, e-learning policy
http://europa.eu/scadplus/leg/en/s19001.htm) are heading toward the requirements based on
multi culture.
4

How to be avoided the handicap of multicultural differences?
Ask what you really taught -This is the simplest way and it gives the most credibility.
Know the facts of your local culture and separate from merely professional facts.
Never forget what to measure: If your target group is international, be aware of the
multicultural elements.
Use short sentences focused on clear, standardized professional expressions what
are learnt and experienced during the teaching process.
Avoid the facts about persons who are well known and taught nation wide, but you
have no evidence if they were well known internationally.
Avoid the facts based on figures described by the inner market, by local history, or by
local media
These seem to be trivial, but if someone made a short online survey in e-learning courses he
could soon find some evaluation details which are not efficient in a multi culture frame yet.
How to measure in international circumstances - that have much more difficulties which are
still uncovered and waiting for research. Generally E-tutors have no enough information
about applicants. The wise decision they could make is to position of e-valuation standards
to the nonnative speakers, to use the generally accepted glossary terms or going further, use
the default basic meaning of the words what could be familiar with nonnative persons speak
English as a foreign language.
Responsibility – Credibility
What to teach and how to teach – that have tradition in the methodology, but Ones who are
responsible for some e-valuation they have to respect the diversity of participants. When
technology transfers contextual values from the cultural context of origin to a different culture,
evaluation are still uncovered.
We have just made the first steps and made the first insight.
Although E-tutors would recommend e-learning software and teaching with a variety of
teaching styles and choosing a number of delivery methods, they could not bridge the lack of
Cultural Elements and of Relative Cultural Distance without the fine selection of educational
facts and figures for international participants. The also have to design e-valuation with a
sophisticated but limited level of language.
The key part of supporting multicultural education is to continuously re-examine all aspects of
education, in light of principals of equity and of diversity.
E-valuation merges traditionally separated areas of teaching across cultures and it will
challenge us to find the quality and the optimal ways to teach and evaluate globally.
My experience is: A reliable syllabus for the design of E-learning tests should have issues of
how to manage multicultural elements within e-valuation process.
I made my own way before start to design a brand new e-learning system based on project
management and Web2.0.
The Euro Bridge English Club System is now under process of accreditation.
Gabriella Lakatos
Senior EFL Teacher
CeM eMA Professional Member
CEO Euro Bridge Hungary Communication Consulting
www.e-bridges.eu
5

Appendix:
Student Traits and Attributes Contributing toSuccess in Online Courses: Evaluation of
University Online Courses
Lori B. Holcomb University of Connecticut
Frederick B. King University of Hartford Scott W. Brown
University of Connecticut
The Journal of Interactive Online Learning Volume 2, Number 3, Winter 2004
A Framework for the Evaluation of E-Learning
Jenny Hughes
CRED - Centre for Research in Educational Development
Graham Attwell
KnowNet, Pontydysgu, and ITB, U. Bremen
Paper presented to a seminar series on Exploring models and partnerships for eLearning in
SMEs, held in Stirling, Scotland and Brussels, Belgium, in Nov 2002 and Feb 2003
Quality Standards in e-Learning: A matrix of analysis The International Review of Research
in Open and Distance Learning, Vol 3, No 2)
http://en.wikipedia.org/wiki/Evaluation
http://www.elearningeuropa.info/
http://europa.eu/scadplus/leg/en/s19001.htm)
http://www.theadvocates.org/quiz.html
2008
6