1) The document discusses a study on Iranian EFL test takers' perspectives on computer-assisted language testing (CALT) at Payame Noor University. The study aims to identify advantages and disadvantages of CALT, and examine the impact of CALT experience and academic degree on perspectives. 2) Key findings include computer anxiety being linked more to issues like hardware/software problems rather than computer literacy. Experience with CALT leads to more positive perspectives as test takers gain familiarity. Academic degree had no influence on perspectives. 3) Recommendations include improving technology infrastructure, providing CALT training, and assessing a wider range of language skills to better align tests with instruction.

1. In the name of God

2. Iranian EFL Test Takers ’Perspectives and Reflections towards
Computer-Assisted Language Testing at Payame Noor University
Master’s Thesis
Supervisor: Dr. Vahid R. Mirzaeian Ph.D
Advisor: Dr. Mohsen Shirazizadeh Ph.D
Presenter: Sedigheh Hajizadeh

3. ➢ The assistance of object-regulations: Frawley & Lantolf (
1995
)
● New testing techniques using the potentials of technology are required to catch up with
ongoing changes in teaching methods in the academic contexts .
● The advent of technology in language learning and testing in Iranian context has
brought facilities & shortcomings.
● Testing delivery mode influences test takers ’performance accordingly it affects
reliability and validity of the test.

4. ➢ The primary goals of the current study:
1. To unfold the advantages and shortcomings of CALT employed at Payame Noor
University from the test takers and teachers ’viewpoints .
2. To investigate the role of exam experience and academic degree on the test takers ’
perspectives towards CALT.
3. To observe the interaction effect between CALT experience and academic degrees, i.e.,
MA and BA.

5. ➢ Theoretical Framework
● Chapelle & Douglas, 2006
;
,yeooD
2008
;
,yekcO
2009
;
vorovuS
&
,remiehlegeH
2014
;
ellepahC
&
,ssoV
2016
;
ellepahC
&
,oruaS
2017
● They offered a number of advantages & shortcomings of CALT regarding the delivery
issues, computer assessment of vocabulary, oral skills, writing, listening, & reading.

6. ➢ Research Design
➢ Exploratory mixed methods: yrotarolpxe(
laitneuqes
)ngised
➢ Qualitative inquiry: observations, interviews, verbal reports
➢ Quantitative study: closed-response questionnaire
● Two-way ANOVA between-groups analysis of variance

7. ➢ Participants :
The students of Payame Noor University
● Purposive sampling for the interview: 6 English instructors with the experience of taking
tests through CALT & 5 students taking the test through computers at least 4 times between
the age range of 21 to 50 male & female
● Simple random sampling for the questionnaire: BA & MA students studying English;
162 participants; hard copies

8. ➢ Procedure:
● Observation: the participants ’behaviors were observed while taking the test through
computers during 42 hours in 12 days.
● Interviews: 12 test-takers, semi-structured
● Closed-response questionnaire: detaler
ot
eht
sgnidnif
gnidulcni
37 items from two
already designed questionnaires.
● The reliability was checked by Cronbach’s alpha coefficient .

9. ➢ Reliability of the Questionnaire
● Values above .7 are considered acceptable & values above .8 are preferable.

10. ➢ Qualitative Research Questions
●
1
: What are Iranian EFL instructors and test takers ’perspectives towards employment of
computers for testing at academic contexts?
●
2
: What are the benefits or challenges of taking tests through CALT rather than PPT based
on the test takers ’experiences?

11. Quantitative Research Questions
● 3
: Does the experience of CALT influence English students ’perspectives?
● 4
: Does the academic degree (MA & BA) affect the students ’perspectives ?
● 5
: Are there meaningful differences in the perspective of inexperienced graduates &
undergraduates with those of experienced graduates & undergraduates?

12. ➢ 288codes, 48 separate categories, 4 themes based on the bottom-up in vivo and open
coding

13. ➢ 1
. Computer Anxiety Experienced through CALT
● English tests at PNU includes multiple-choice or yes-no questions requiring the test takers
to recall the receptive knowledge and to fill in the blanks.
● These types of questions do not exploit cognitive capacity or working memory, hence, they
do not affect the examinees ’performance on simplified CALT.

14. ➢ 1
. Computer Anxiety Experienced through CALT
● The common anxiety felt by the test takers of PNU is not ascribed to any computerphobia
but to other factors such as;
● Suspicion to the quality of computers employed at PNU
● Failure in maintaining the data during the exam
● Disabling to enter the exam portal due to delay
● Failure in submitting the test successfully

15. ➢ The Root of Computer Anxiety Experienced through CALT
● Hardware and software infrastructure: inappropriate supplies, power outage
● Required technical support: adequacy of the number of technical staff, qualified proctors
to control any flaws
● Flexibility of test method: strict time limitation to enter the test portal
● Auto-saving software: gnisol
eht
atad
eud
ot
yna
detcepxenu
sehsarc

16. ➢ 2
. Experience of Computerized Exam Underpinning Computer Literacy
● Examinees ’computer literacy should not be taken for granted.
● Preliminary knowledge about CALT is sufficient to answer MCQ.
● More computer familiarity & experience of CALT reduce computer anxiety.
● Experience of CALT & basic computer literacy enhance computer self-efficacy.
● Reducing experience-related diversity leads to equality among computer literate &
computer illiterate examinees.

17. ➢ 3
. CALL Prerequisite or Completing CALT
● CALL is fitted perfectly with the policy of PNU regarding education for everyone at any
time and anywhere.
● Holding workshops and adding CALL in syllabus are required to establish reliable
instructions for computer self-efficacy.
● Employment of CALL as supplementary for CALT facilitates changes of old test habits
from PPT to CALT.

18. ➢ 4
. Challenges vs. Opportunities Brought by CALT
● Environment-friendly
● Immediate and auto-scoring
● Cost-effectiveness in long run
● Eliminating paper consumption
● Less time and energy consumption
● Reduction in human source for preparation, distribution, administration

19. ➢ 4
. Challenges vs. Opportunities Brought by CALT
● The need for a comprehensive & auto-saving program
● The need for qualified staff to control the drawbacks
● The need for assessing productive skills
● Cost of equipment and maintenance
● Negative washback effect
● Challenge of old habits

20. ● The variability of scores across groups is equal; . 363
>
.
05

23. ● 3
. The main effect for independent variable Experience: F ( 1
,
159
(
=
16.85
,
p
0.01
<
P
.
5
,
eht
N
0 was rejected. CALT experience had a significant effect on examinees ’perspectives
towards CALT. The higher the experience, the more positive perspectives towards CALT.
● 4
. The results of academic degree is Degree: F ( 1
,
159
=(
.
020
,
p
=
0.886
>
P
.
5
,
eht
N
0
remains. The academic degree (MA & BA) had no influence on the examinees ’perceptions
regarding CALT .
● 5
. The interaction effect between experience & degree was not significant Experience *
Degree: F ( 1
,
159
(
=
.
296
,
p
=
0.597
>
.
05
.
,
● In every situation experience makes a significant effect.

24. ● Examines mostly get low scores due to the lack of CALT experience; high anxiety, and old
testing habits.
● To reduce anxiety towards CALT requires to hold workshops and to take the experimental
exams.
● The test takers ’perspectives moderated by taking more exams through computers as a
result of gaining experience.
● The more proficient and experienced on computerized testing, the more inclined toward
CALT.

25. ● The shortcomings of CALT lead to construct under-representation & construct irrelevant-
variance, resulting test score inflation or deflation in favor of one group & detriment to the
other group.
● Testing only receptive skills through MCQ can not tap the intended construct hence causes
changing in learning style, studying one night before exam, getting a pass mark not mastery
of materials & washback effect .
● The impact can be eliminated by “changing the testing method in which the characteristics
of tests correspond to the characteristics of the instructional program ”
.

26. ● Any human or computer errors affect the students ’lives, waste their time and energy, and
minimize test reliability or validity.
● The threats of CALT can be prevented by:
● Enhancing quality of hardware infrastructures
● Designing auto-saving programs
● Increasing technical support

27. ● Designing solid & auto-saving programs
● Maintaining software & hardware supplies
● Reassuring technical support through the whole exam
● Holding CALT workshops for both the students & proctors
● Empowering English students with basic computer literacy, e.g., CALL
● Reducing the chance of washback by changing the way of testing
● Developing the tests for all skills both receptive and productive ones
● Providing equal condition to prevent any unfair inflation/deflation of scores

28. Thank you for your attention
Thank you for your attention