Presentation by Jean-Claude Callens, Vives University at the 2018 European Distance Learning Week's third day webinar on "Innovative Education – Case Studies" - 7 November 2018 Recording of the discussion is available: https://eden-online.adobeconnect.com/pynq0w4ku2b1/

1. Evaluation in distance education @ VIVES:
some tools and trends.
JC Callens

2. 1. Introduction
• Distant education in VIVES: Start 1997
– Teacher education
– 7 students
• Today
– 30 programs: n = 2300 students
– 5 study areas with distance education
• Commercial sciences, business management and informatics
• Heath care
• Applied social studies
• Education
• biotechnology
– 15 % percent of VIVES student population

3. • a smaller number of students that follow a study by distance
education get their degree high drop-out
– Cf. students who follow a f2f program
• There is the challenge to increase the study efficiency in
distance education
Focus: how reduce high drop-out in distance education?
How to enhance retention?

4. Starting point: student profile
Students often opt for distance education as it allows for the
combination of work/care of family and study.
Work * Life*balance* participant Crosstabulation
Family?
Total
Single
without
children
With
partner
without
children
Single with
children
With
partner
and
children other
Work? fulltime Count 54 94 20 118 4 290
% of
Total 6,6% 11,5% 2,5% 14,5% 0,5% 35,5%
50-90% Count 53 66 13 155 11 298
% of
Total 6,5% 8,1% 1,6% 19,0% 1,3% 36,5%
Less than 50% Count 21 19 2 45 13 100
% of
Total 2,6% 2,3% 0,2% 5,5% 1,6% 12,3%
unemployed Count 25 23 5 55 20 128
% of
Total 3,1% 2,8% 0,6% 6,7% 2,5% 15,7%
Total Count 153 202 40 373 48 816
% of
Total 18,8% 24,8% 4,9% 45,7% 5,9%
100,0
%

5. weisure
Combination of Work & Leisure Weisure
Cf. control over time, place & pace is important
And thus learner control

6. Weisure  learner control
In summary, possible advantages of learner control
• Combination work/care for family and study  weisure (Dickinson, 1995)
• More motivation (Scheiter & Gerjets, 2007; Dickinson, 1995; Kirschner, 2002)
• Adaptative cognitive needs (Scheiter & Gerjets, 2007)
• More elaboration (Daniels, 1996)
For us: Learner control and distance education are like Laurel & Hardy
Source: http://beertjes.blogspot.com/2004_08_01_archive.html

7. And for the Flemish
distance education and learner control are like
‘Nicole en Hugo’

8. Flexibilization of evaluation
• Students who follow a distance education program are still
often expected to get to campus at fixed times; for instance to
take exams.
• Challenges
– Take the exam at a time that best suits the students
– Take written exams remotely

9. In summary:
reduce drop-out through
learner control
flexibilization of evaluation place and time

10. 3. If it walks and talks like a duck … it’s a duck!
• Analysis of ‘evidence’
– analysis of literature
– data-analysis

11. 3.1 Research and learner control?
In summary
“I have reviewed is that it would be a mistake to interpret the current
constructivist view of learning as a rationale for reviving pure discovery as a
method of instruction. Pure discovery did not work in the 1960s, it did not work in
the 1970s, and it did not work in the 1980s, so after these three strikes, there is
little reason to believe that pure discovery will somehow work today.” (Mayer,
2004, p.18)
1) People need a ‘nudge’ (Thaler & Sunstein, 2008)
Nurture by structure
2) Prior knowledge is important

12. 3.2 Data-analysis
• Focus in this contribution: Learner control with regards to
assessment in distance education through
– Flexibilazation of time
– Flexibiliazatioin of place

13. 3.2.1 Flexibilization of time
Degree of flexi
Between-Subjects Factors
Value Label N
opdeling aantal momenten
item in drie categorieën
(meest/minst)
1 meest (cf. 1/3) 1198
2 midden (cf. 1/3) 1286
3 minst (cf. 1/3) 1155
Tests of Between-Subjects Effects
Dependent Variable: nummerieke score die de student krijgt voor examen/opdracht/...
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 726,248a 2 363,124 21,733 ,000
Intercept 532471,121 1 532471,121 31867,909 ,000
CategorieMomentenItem 726,248 2 363,124 21,733 ,000
Error 60752,810 3636 16,709
Total 595137,573 3639
Corrected Total 61479,058 3638
a. R Squared = ,012 (Adjusted R Squared = ,011)

14. Impact student characteristics
Impact time before graduation
Between-Subjects Factors
Value Label N
Afstudeertermijn 1 in 1 jaar 288
2 in 2 jaar 1009
3 in 3 jaar 709
4 meer dan 3 jaar 1367
opdeling aantal momenten
item in drie categorieen
(meest/minst)
1 meest (cf. 1/3) 1110
2 midden (cf. 1/3) 1207
3 minst (cf. 1/3) 1056
Tests of Between-Subjects Effects
Dependent Variable: nummerieke score die de student krijgt voor examen/opdracht/...
Source
Type III Sum of
Squares df Mean Square F Sig.
Corrected Model 1497,193a 11 136,108 8,196 ,000
Intercept 322354,699 1 322354,699 19410,831 ,000
Afstudeertermijn 603,058 3 201,019 12,105 ,000
CategorieMomentenItem 393,844 2 196,922 11,858 ,000
Afstudeertermijn *
CategorieMomentenItem
336,202 6 56,034 3,374 ,003
Error 55815,959 3361 16,607
Total 550756,798 3373
Corrected Total 57313,152 3372
a. R Squared = ,026 (Adjusted R Squared = ,023)

15. 3.2.2 Flexibilization of place
• Challenge
– ‘Take written exams remotely’

16. Systematic search in literature
• Databases
– ‘Academic search premier’, ‘Business source elite’, ‘ERIC’,
‘Science direct’, ‘Springer’, ‘Web of science’, and ‘Wiley
online library’ was done (november 2015).
• The search strategy in this study focused on the keywords
‘higher education’, ‘assessment’, ‘remote(ly)’, ‘and ‘online
proctored’.
• As inclusion criteria, only peer-reviewed, published after 2010
and full-text online were included.
• Only 2 articles of the 91 hits are selected based on the criteria
whether or not they contain a description of remotely written
examination in higher education.

17. The first study (Lievens & Burke, 2011) reveals that
• 1) there is little information available about unproctored
internet testing (UIT) and
• 2) no significant differences between proctored and
unproctored internet testing could be retrieved regarding the
evaluation scores.

18. The results of the second study (Jordan, 2011)
show that
• 1) student perceptions seems to be important during the
implementation of an interactive computer-based
assessment;
• 2) assessments are often poorly designed;
• 3) the introduction of new forms of educational technology
may lead to the further excluding of the already socially
excluded, and
• 4) feedback after an assessment is important.

19. Tool to organize assessment remotely in VIVES
Proctorexam
• Proctorexam combines webcam, screen sharing, and video
recording which allows to monitor the student remotely while
an exam is taken.
• https://www.vives.be/ontdek-onze-examenmomenten

20. Methodology: focus on perceptions of students
• The questionnaire was processed through limesurvey. First there were some
questions about personal characteristics (see description participants); next
the following questions/statements were offered.
– What is the reason that you choose to take an exam from a remote
location? (answer alternatives with this question were: distance to
campus/you can choose the time when you take your exam/a combination
of the two alternatives/other reasons)
– I experienced a remote exam as stressful. (1) completely false; 2) false; 3)
true; 4) completely true)
– To take an exam from a remote location is less/more stressful than an
exam on-campus. ( 1) completely false; 2) false; 3) true; 4) completely
true)
– Would you take a remote exam again? (yes/no)
– What was good? What can be improved? (open question)
– Which support was okay and which support do you additional want?
(open question)
– Other remarks? (open question)

21. Participants: 37 students

22. Results (explorative study!!)
Students
Reason?
• 17 indicate the combination of the distance to the campus and the opportunity to control the time they
can take an exam.
• Other reasons were: to limit the number of times they have to drive to the campus, so the cost for
gasoline is reduced (=1), trying an experience (=1).
Stress?
• 16 of the 37 participants perceive an exam remotely as stressful,
• 22 participants find taken an exam from a remote location less stressful than an exam on campus.
cross tabulation shows that the same participants whom perceive an exam taken from a remote location
as stressful, also find a remote exam more stressful than an exam on campus.
Other?
• Almost all participants will take a remote exam again (=37 of the 38 participants).
• Two students would like to have the opportunity to ask question during the exam; in case of ambiguities
about the content of the exam.
• Further, two other students want more technical support,
• one student mentioned that a timer would help.
• There was the remark (one student) that it feels strange not to see or hear anyone during the exam.
Time?
• Students choose distinct moments to take a remote exam: in the morning, around noon, in the late
afternoon and in the evening. Based on the results of this small population there is no preference of the
participants to choose a particular day to take an exam from a remote location.

23. 4. In summary
• Reduce drop-out in distance education?
• Organization of evaluation
– More flexibility beter results
• Support students with less prior knowledge
– Explorative study

24. • Contact: jean-claude.callens@vives.be
Thanks!