1. IEEE TRANSACTIONS ON EDUCATION, VOL. 48, NO. 2, MAY 2005 1
Student Online Assessment Behaviors
Thomas G. Cleaver, Senior Member, IEEE, and Loay M. Elbasyouni, Student Member, IEEE
Abstract—Interactive online tutorials are used in an intro-
ductory circuits class as an adjunct to traditional educational
delivery. The online behavior of the students was monitored using
Web-based parameter-passing strategies and cookies. The authors
found that most students repeated the tutorials a sufficient number
of times to get perfect scores.
Index Terms—Automated grading, educational technology, stu-
dent behavior, tutorials, Web-based education.
I. INTRODUCTION
EDUCATORS increasingly rely on the Web for educational
delivery. In addition to syllabi, reading assignments, and
hyperlinked course content, many Web-enabled courses pro-
vide advanced features, such as interactive tutorials [1], auto-
matically graded homework [2], online readiness quizzes [3],
and online tests [4]. Analyzing the online behavior of students
when they perform such tasks is crucial to an instructor’s under-
standing of how they use the Web to learn.
This study examines student online behavior in solving inter-
active tutorials. Conclusions reached may help guide instructors
in their choices of Web-based educational materials and their re-
liance on these materials for student assessment.
II. NETWORK ANALYSIS TUTORIALS
ECE 220 Network Analysis I is the first course in circuits
for students at the School of Engineering of the University of
Louisville, Louisville, KY. In addition to the usual in-class
homework, quizzes, and tests, there are extensive Web activities.
These activities include a set of 20 interactive tutorials to rein-
force the students’ in-class lessons. These tutorials cover topics
from Kirchhoff’s laws to balanced three-phase systems. They
were created by the first author for use in his classes; in addition,
they are published by McGraw-Hill for instructors and students
who use McGraw-Hill textbooks (for samples of the tutorials,
please see http://www.mhhe.com/engcs/alexander2e/netan_
tutorials/tutorials/tutmenu.htm).
The development, content, function, and effectiveness of
these tutorials have been reported elsewhere [5]–[8].
The tutorials are designed to guide the student through the
solution of typical problems and to assess their progress. Tuto-
rials consist of linked HTML pages, each of which requires the
student to enter an answer. As a student goes through a tutorial,
the student types in answers, receives context-sensitive hints in
a feedback window, and navigates back and forth through the
Manuscript received June 21, 2004; revised December 1, 2004. This work
was supported in part by the University of Louisville, Louisville, KY, under an
Undergraduate Research Grant.
The authors are with the Department of Electrical and Computer Engineering,
University of Louisville, Louisville, KY 40292 USA.
Digital Object Identifier 10.1109/TE.2004.842886
tutorial. If the student exhausts all possible hints without get-
ting the correct answer, the page is marked as incorrect, and the
student is given the answer and asked to enter it in the answer
space. A correct answer must be given before the student is al-
lowed to proceed to the next page.
When a student begins a tutorial, the student must select
between taking the tutorial for practice (as an anonymous user)
or credit. In the latter case, the student’s name and e-mail
address must be entered. Each student’s progress through the
tutorial is monitored by capturing variables such as time,
answers given, hints accessed, and graphical user interface
(GUI) buttons clicked. At the end of the tutorial, the results
are automatically e-mailed to the instructor. A student may
choose to restart the tutorial or simply exit the program before
the e-mail is sent. The e-mail contains detailed information
on the last session (answers given, hints accessed, etc.) and
rudimentary information on aborted sessions (buttons clicked
and the times when they were clicked).
The goal was to learn what strategies students employed when
they worked on the tutorials in order to improve the tutorials and
to assess the online methods that affect student learning.
III. PROCEDURE
In the spring semester of 2003, data were gathered from the 22
students in ECE 220 Network Analysis I. Most students did all
20 tutorials. A few skipped some tutorials, while others repeated
tutorials, some as anonymous users.
In a typical tutorial session, the student will begin the lesson
as an anonymous user, and upon completion restart the lesson as
an identified user. Only limited data are available for the anony-
mous session (times and pages visited), but more detailed data
are available for the session in which the user was identified by
name. These data include the time that each page was visited,
the answer given for each page, the number of hints given for
each page, and the score.
Over 700 e-mails with student tutorial data were generated
during the term. The data in these e-mails were captured and an-
alyzed with a spreadsheet program (Microsoft® Excel), which
allowed the data to be analyzed with respect to number of at-
tempts of each tutorial, number of hints given, score statistics,
final session durations, and total session durations. The data
could also be examined to identify typical patterns of behavior.
Students were told that the tutorials would count for 10%
of the course grade. Students were allowed to repeat a tutorial
as often as they wished, prior to the deadline, to improve their
grades.
IV. RESULTS
Scores on the tutorials were quite high. The average score for
all tutorials was 98.65, with a standard deviation ( ) of 9.09.
0018-9359/$20.00 © 2005 IEEE

2. 2 IEEE TRANSACTIONS ON EDUCATION, VOL. 48, NO. 2, MAY 2005
All students scored 100% for nine of the 20 tutorials. Out of
a total of 438 scores recorded, 416 were 100% correct. One
might think that the high scores indicate that the tutorials are too
easy, but another explanation is available. The average number
of attempts for all tutorials was 2.20, with a of 1.45. This re-
sult means that the average student tried each tutorial more than
once. This finding implies that many students try each tutorial
repeatedly before submitting the tutorial for grading. Further
evidence for this conclusion is given in data for time spent on
each tutorial. The average for the total duration (the duration for
all attempts) was 22.03 min, with a of 22.50, while the final
duration (the duration for the scored attempt) was 6.75 min, with
a of 7.49. Note that the total duration is significantly longer
than the final duration. The implication of the data is that most
students repeat tutorials until they get perfect scores; that is, they
seek mastery.
One way a student might seek to get a perfect score is by
going through the tutorial anonymously to get the correct an-
swers, writing them down or memorizing them, and then re-
peating the tutorial for credit. Another valid explanation is that
students learn how to solve the problems in earlier attempts and
correctly solve them on the final attempt. Both behaviors prob-
ably occur in the student population.
V. CONCLUSION
The most striking conclusion that can be drawn from this
study is that students taking these online tutorials seek mas-
tery, although one may argue that a sample size of 22 students is
insufficient to be confident of this conclusion. In general, they
will repeat a tutorial as many times as necessary to get a perfect
score. Whether they are motivated by a desire to learn the ma-
terial or the desire to get a high grade is not clear. Regardless,
instructors may be able to capitalize on this behavior to improve
online learning.
Doing a tutorial quickly to get the answers, and then repeating
it to enter the (memorized) correct answers, may be a problem.
The data are insufficient to determine how prevalent this strategy
is. Providing tutorials with randomized parameters seems a wise
alternative to tutorials with fixed answers (implemented with six
of the 20 tutorials). Not only are the effects of memorizing an-
swers mitigated, but students have the opportunity to repeat tuto-
rials for additional practice. Creating tutorials with randomized
parameters can be significantly more work for the instructor, but
the effort may be worthwhile.
In anecdotal reports, students praise the effectiveness of the
online tutorials and rate them highly as an efficient component
of their learning. This evidence as well as the more objective
evidence indicates that online tutorials can be an important part
of engineering education.
REFERENCES
[1] J. R. Jones and D. A. Conner, “The development of interactive tuto-
rials for introductory circuits,” in Proc. 1st Int. Conf. Multi-Media En-
gineering Education, IEEE Multimedia Engineering Education, Mel-
bourne, Australia, Jul. 6–8, 1994, pp. 108–109.
[2] S. Hsu, “HWSAM: A web-based automated homework submission
system,” in Proc. 28th ASEE/IEEE Frontiers in Education Conf. (FIE
1998), Tempe, AZ, Nov. 4–7, 1998, pp. 580–582.
[3] B. P. Marks, “Web-based readiness assessment quizzes,” J. Eng. Educ.,
vol. 91, no. 1, pp. 97–102, 2002.
[4] V. Mornar, N. Hoic-Bozic, and D. P. Zokovic, “Approaches to online
testing in web-based educational systems,” in Proc. EUROCON 2003,
Ljubljana, Slovenia, Sep. 22–24, 2003, pp. 343–346.
[5] T. G. Cleaver. (1999, Jul.). “ALN in a small on-campus engineering
class” [Online], vol (1). Available: http://www.sloan-c.org/publica-
tions/magazine/v3n1/cleaver.asp.
[6] , “Interactive web-based tutorials for engineering education,” in
Proc. IEEE SOUTHEASTCON ’99, Mar. 1999, pp. 126–127.
[7] , “Design of a web-based education environment,” in 29th
ASEE/IEEE Frontiers in Education Conf. (FIE 1999), Nov. 1999, pp.
12A3 1–5.
[8] , (2002, Nov.) Online assessment with automated e-mail reporting.
Int. Online Conf. Teaching Online in Higher Education (TOHE) [On-
line]. Available: http://www.ipfw.edu/as/2002tohe/
Thomas G. Cleaver (S’61–M’63–SM’74) received the B.S.E.E. degree from
Case Institute of Technology (now Case Western Reserve University), Cleve-
land, OH, in 1963 and the M.S. degree in electrical engineering and the Ph.D.
degree in biophysics from Ohio State University, Columbus, in 1966 and 1969,
respectively.
He is a Professor in the Electrical and Computer Engineering Department of
the University of Louisville, Louisville, KY. His research focuses on educational
technology.
Dr. Cleaver is a licensed Professional Engineer in the Commonwealth of Ken-
tucky.
Loay M. Elbasyouni (S’98) received the B.S. degree in electrical engineering
from the University of Louisville, Louisville, KY, in 2004. He is now pursuing
the M.S. degree at the same institution.
He is a Microsoft Certified Software Engineer (MCSE).