2. PRINCIPLES FOR CALL
EVALUATION2
Three needs must be addressed when
evaluating CALL materials:
Materials include
Courseware – CD ROMs etc
Websites – ESL website
Mobile Apps -
Tasks & – assignments including use of WWW, CMC
Activities – Chat, Blog, Podcast, etc
3. PRINCIPLES FOR CALL
EVALUATION3
Evaluation criteria should incorporate
findings and theory-based speculation about
ideal conditions for SLA
Findings from research on SLA
Theories of SLA
4. PRINCIPLES FOR CALL
EVALUATION
Criteria should be
accompanied by
guidance as to
how they should
be used i.e. a
theory of
evaluation needs
to be articulated
4
5. PRINCIPLES FOR CALL
EVALUATION5
Both criteria and theory need to apply not only
to software, but also to the task that the
teacher plans and that the learner carries out
6. PRINCIPLES FOR CALL
EVALUATION6
According to Chapelle (2001)
5 principles of CALL evaluation criteria
Evaluation of CALL is a situation-specific argument.
CALL should be evaluated through two perspectives: judgmental
analysis of software and planned tasks, and empirical analysis of
learners’ performance.
Criteria for CALL task quality should come from theory and
research on instructed SLA.
Criteria should be applied in view of the purpose of the task.
Language learning potential should be the central criterion (most
critical) in evaluation of CALL.
7. CALL EVALUATION:
PRINCIPLE 17
Evaluation of CALL is a situation-specific argument.
The outcome of task evaluation for any L2 tasks including those for CALL cannot be a
decision about effectiveness.
Instead, an evaluation has to result in an argument indicating in what ways a particular
CALL task is appropriate for particular learners at a given time, the use at a particular
setting
Implication: CALL developers need to be familiar with criteria for evaluation which
should be applied relative to a particular context.
8. CALL EVALUATION:
PRINCIPLE 28
CALL should be evaluated through two perspectives: judgmental
analysis of software and planned tasks, and empirical analysis of
learners’ performance.
Judgmental analyses - examine characteristics of the software and
task in terms of criteria drawn from research on SLA.
Empirical analyses - address the same criteria but through
data gathered to reveal the details of CALL use and learning
outcomes.
Implication: CALL developers need to be familiar with criteria
for evaluation which should be applied relative to a particular
context.
9. LEVELS OF ANALYSIS FOR CALL EVALUATION
9
Level of
Analysis
Object of Evaluation Example Question Method of
Evaluation
1 CALL software
Does the software provide
learners the opportunity for
interactional modifications to
negotiate meaning?
Judgmental
2 Teacher-planned
CALL activities
Does the CALL activity
designed by the teacher
provide learners the
opportunity to modify
interaction for negotiation of
meaning?
Judgmental
3
Learners’
performance
during CALL
activities
Do learners actually interact
and negotiate meaning while
they are working in a chat
room?
Empirical
10. CALL EVALUATION:
PRINCIPLE 310
Criteria for CALL task quality should come from theory and research on instructed SLA.
Language learning potential
Learner fit
Meaning focus
Authenticity
Positive impact
Practicality
Implication: CALL evaluators need to keep up with and make
links to research on instructed SLA.
11. CRITERIA FOR CALL TASK
APPROPRIATENESS
11
Language
Learning Potential
•The extent to which the activity can be considered to be a language learning activity rather than simply an opportunity for language use.
•The degree of opportunity present for beneficial focus on form.
Learner Fit
•The individual differences in linguistic ability level and non-linguistic characteristics.
•The amount of opportunity for engagement with language under appropriate conditions given learner characteristics.
Meaning Focus
•The learner’s primary attention is directed toward the meaning of the language that is required to accomplish the task.
•The extent to which learners’ attention is directed toward the meaning of the language.
Authenticity
•The degree of correspondence between an L2 learning task and tasks that the learner is likely to encounter outside the classroom
•The degree of correspondence between the CALL activity and target language activities of interest to learners out of the classroom.
Positive Impact
•Its effects beyond its language learning potential. E.g. – help build metacognitive strategies, willingness to seek out opportunities to use TL.
•The positive effects of the CALL activity on those who participate in it.
Practicality
•How easy it is for the learners and teachers to implement a CALL task within the particular constraints of a class or language program.
•The adequacy of resources to support the use of the CALL activity.
12. CRITERIA FOR CALL TASK
APPROPRIATENESS
12
Language
Learning
Potential
• They learn e.g. how to use modals
Learner Fit
• Task appropriate at their level, taking into also consideration age, learning style, willingness to communicate
Meaning Focus
• Meaning of language to accomplish a task. e.g. using language to make decision, exchanging information to
accomplish a goal
Authenticity
• What they would actually encounter outside of the classroom, in the real world
Positive Impact
• E.g. Seek opportunities to use what they have learned in class outside.
Practicality
• Easy to implement CALL despite lacking facilities
13. CALL EVALUATION:
PRINCIPLE 413
Criteria should be applied in view of the purpose of the
task.
Implication: CALL tasks should have a clearly
articulated purpose.
14. CALL EVALUATION:
PRINCIPLE 514
Language learning potential should be the central
criterion (most critical) in evaluation of CALL.
Implication: Language learning potential
should be one aspect of the purpose of CALL
tasks.
15. JUDGEMENTAL ANALYSIS OF
CALL APPROPRIATENESS15
Qualities Questions
Language learning
potential
Do task conditions present sufficient opportunity for beneficial focus on form?
Learner fit Is the difficulty level of the targeted linguistics forms appropriate for the learners to
increase their language ability?
Is the task appropriate for learners with the characteristics of the intended learners?
Meaning focus Is learner’s attention directed primarily toward the meaning of the language?
Authenticity Is there a strong correspondence between the CALL task & second language tasks of
interest to learners outside the classroom?
Will learners be able to see the connection between the CALL task and tasks outside the
classroom?
Impact Will learners learn more about the target language and about strategies for language
learning through the use of the tasks.
Will instructors observe sound second language pedagogical practices by using the task?
Will both learners and teachers have a positive learning experience with technology
through the use of the task?
Practicality Are hardware, software, and personnel resources sufficient to allow the CALL task to
succeed?
16. Empirical Analysis of CALL
Appropriateness16
Qualities Questions
Language learning
potential
What evidence suggests that the learner has acquired the target forms that
were the focused on during the CALL tasks/
What evidence indicates that learners focused on form during the CALL task?
Learner fit What evidence suggests that the targeted linguistic forms are at an
appropriate level of difficulty for the learners?
What evidence suggests that the task is appropriate to learners’ individual
characteristics (age, learning styles, computer experience)
Meaning focus What evidence suggests that the learner’s construction of linguistic meaning
aids language learning?
What evidence indicates that the learners use the language during the task
for constructing and interpreting meaning?
Authenticity What evidence suggests that learners’ performance in the CALL tasks
corresponds to what one would expect to see outside the CALL tasks?
What evidence suggests that the learners see the connection between CALL
task and tasks outside the classroom?
Impact What evidence suggests that the learners learn more about the target
language and about strategies for language learning through the use of the
tasks?
Practicality What evidence suggests that hardware, software and personnel resources
prove to be sufficient to allow the CALL task to succeed?
