(2006) An assessment of performance between on and off-campus students in an …


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

(2006) An assessment of performance between on and off-campus students in an …

  1. 1. AN ASSESSMENT OF PERFORMANCE BETWEEN ON- AND OFF- CAMPUS STUDENTS IN AN AUTOMATIC IDENTIFICATION AND DATA CAPTURE COURSE Stephen J. Elliott1; Eric P. Kukula 2, & Nathan C. Sickler3 Abstract  Universities have been exploring distance Distance education as a supplement to classroom education for years, but typically do not offer program instruction is common practice in many traditional higher- courses that require laboratory exercises. Had they education institutions. Instructors deliver classroom offered such courses, off-campus (distance) students lectures, but homework assignments and supplemental would have been academically disadvantaged by the material may be retrieved online. This approach is inability to participate in hands-on laboratory exercises. feasible because these institutions have sufficient numbers Therefore, an undergraduate course in Automatic of on-campus computers with Internet access. The Identification and Data Capture was designed to residence halls typically accommodate students’ personal accommodate distance students and ensure a laboratory computers by offering in-room Internet access, while experience comparable to on-campus students. The students without personal computers can avail themselves increased availability of technological advances of on-campus computer laboratories. (broadband Internet) provides an opportunity for distance Distance education as a stand-alone form of students to gain comparable knowledge. This paper instruction is not as common, but its usage has grown outlines the experiences of three groups of students tremendously in the past ten years; note the flourishing completing laboratory experiments: on-campus students, numbers of online institutions offering degree programs. undergraduate distance students, and graduate distance Many students choose this type of education because full- students. The results showed that students who interacted time jobs, other commitments, or physical limitations with equipment, experience the same level and extent of prevent their participation in more traditional approaches learning, whether interaction with the AIDC equipment to instruction. This approach requires the student to have occurred on-campus or remotely; but a lack of interaction a reliable computer and Internet connection. In addition, resulted in test scores that were statistically different. providing a new channel for homework and learning opportunities not limited by time or location is required. This approach to education offers numerous benefits: Index Terms  curriculum development, automatic identification and data capture, distance education • Accommodates different learning styles and schedules INTRODUCTION • Uses various educational resources or media (e.g., paper-based, video, audio, online material) as Distance education is popularly understood to instructional tools mean the planned learning or educational processes that • Allows usage of multiple communication methods occur with the instructor and student separated either geographically or in time, requiring special methods for (e.g., e-mail, teleconference, video conference, course design, instruction, communication, and instant messaging) administrative tasks [1-2]. Distance education is often • Supports self-directed and self-paced learning style used in two ways: and method [1-5]. • As a supplement to brick-and-mortar (on-campus) It is common to consider distance education in instruction two distinct categories of instruction: synchronous and asynchronous [2,5]. Synchronous instruction focuses on • As a stand-alone form of instruction group work at a distance. It requires all students and instructors to interact simultaneously. The advantage of 1 Stephen Elliott, Ph.D., Assistant Professor, Biometrics Standards Performance, and Assurance Laboratory, Department of Industrial Technology, Purdue University, 401 North Grant Street, West Lafayette, IN 47906, USA, elliott@purdue.edu 2 Eric Kukula, Research Assistant, Biometrics Standards Performance, and Assurance Laboratory, Department of Industrial Technology, Purdue University, 401 North Grant Street, West Lafayette, IN 47906, USA, kukula@purdue.edu 3 Nathan Sickler, Research Assistant, Biometrics Standards Performance, and Assurance Laboratory, Department of Industrial Technology, Purdue University, 401 North Grant Street, West Lafayette, IN 47906, USA, sicklern@purdue.edu ©2006 WCCSETE March 19 - 22, 2006, São Paulo, BRAZIL World Congress on Computer Science, Engineering and Technology Education 93
  2. 2. synchronous instruction is that it occurs in real time, well as laboratory exercises – both the presentation and giving participants the “feel” of a traditional classroom; the content of the module components had to be questions can be asked and answered in an immediate scrutinized. A review of literature showed that there were manner, and the class interacts as a group. A disadvantage a number of ways to provide laboratory experiences to of synchronous instruction is that it requires everyone to distance students, including LabVIEW™, Virtual be available at the same time, as well as have access to Network Computing software [7,8], or providing the the necessary communication and / or visualization software on a CD-ROM for students to execute at home. infrastructure, which may not be feasible due to A combination of the latter two strategies was chosen. technological, financial, or scheduling constraints. Asynchronous instruction, on the other hand, is flexible. It Lecture Material allows students to choose their own instructional time To provide students with the lecture material, frame and complete learning materials according to their and to ensure that both distance and on-campus students schedules [2,5]. Asynchronous instruction allows for received the same lecture material, each on-campus multiple learning levels and schedules, as this category lecture was video-taped, digitized, and posted to a makes use of video- or audio-taped lectures, e-mail, and streaming server. Additionally, distance students were Internet-based programs, all of which are time- provided with a CD that contained the software necessary independent [2,4,5]. Additionally, disciplines that to remotely connect to designated on-campus host structure courses by supplementing lectures with computers, as well as software or videos for specific laboratory exercises offer enhancements in the way of laboratory exercises (see discussion below). For a more related lecture material [6]. detailed description on the course material see [9]. Equipment requirements TRANSFORMATION OF THE LABORATORY INTO All the laboratory activities were evaluated to A VIRTUAL ENVIRONMENT ensure their remote accessibility via the Internet. There were three categories of laboratory activity remote General Course Structure accessibility: Initially, all of the lecture assignments, • The laboratory activities were equivalent (no laboratory assessments, and exams in IT 345 Automatic differences were detected between the coursework Identification and Data Capture (AIDC) were exclusively that distance and on-campus students would paper-based. When the university adopted the use of complete) WebCT Campus Edition™ course software, additional • The laboratory was not convertible (i.e., students had material was developed to take advantage of the new to physically interaction with equipment) technology. For the fall 2003 semester, the instructors migrated the course to a truly online environment • The laboratory was converted with some (WebCT Vista™), dividing the course into modules, each modifications. of which covers an individual AIDC technology. Course material was available electronically; tests were graded Table 1 shows the results of the assessment of the automatically by the course management system, when course’s various laboratory activities. applicable. The WebCT Vista™ portal was instrumental in the department’s ability to offer this course to distance TABLE I CONVERTIBILITY OF LABORATORY ACTIVITIES FOR DISTANCE students; the technology made it possible for distance STUDENTS students to complete and submit material at their Convertible Not convenience. Nevertheless, there remained two main Laboratory Activity Equivalent Convertible with issues to be addressed: Modifications IT 345 On Line Pre- X Test • How would the instructor present lecture material? POSTNET Bar Code X • How would the instructors deliver and allow students — Practical to complete the laboratory activities? Data Density — Post- X Test Linear Bar Code — X Adapting the Course Post-Test Each of the AIDC technology modules was PDF 417 — Practical / Post-Test X assessed to see whether any modification would be Data Matrix — needed to enable distance students to participate fully in Practical / Post-Test X the laboratory activity. The modules included lectures as Verification — X ©2006 WCCSETE March 19 - 22, 2006, São Paulo, BRAZIL World Congress on Computer Science, Engineering and Technology Education 94
  3. 3. Not Convertible Performance measures examined included pre-test scores, Laboratory Activity Equivalent with total module scores, module scores of material not Convertible Modifications Practical / Post-Test requiring laboratory activities (no laboratory required), module scores of material supported by laboratory Verification — Color Post-Test X activities that could NOT be modified for distance Configuration Lab — (laboratory unconvertible), module scores of material Argox Practical X supported by laboratory activities that could be modified Configuration Lab — for distance (laboratory with distance modification), as Quadrus Practical X Configuration Lab — shown in Table 1, and combined average exam scores. Depth-of-Field Additionally, all non-zero scores were used in the Practical X calculations. Zero scores resulting from students missing iButton® — Practical / exams, assignments, or assessments were removed to Post-Test X prevent the artificial lowering of any group’s Magnetic Stripe — Practical / Post-Test X performance. RFID Evaluation — Practical X Pre-test scores Voice Recognition — X Post-Test An analysis of each group’s pre-test scores was Hand Recognition — X performed to determine if any statistically significant Post-Test difference existed between the groups’ starting knowledge Hand Software — PT2 of the course material. The on-campus students had a Assessment X Face Recognition — mean score of 735.3 and median of score of 736 Cognitec Checkoff X (approximately 41 percent for mean and median).The Face Recognition — X distance students had a mean score of 714.6 and median Post-Test score of 720 (approximately 40 percent for mean and Fingerprint — Post- X median). A comparison of the means and medians proved Test that outliers were not influencing the mean score for Fingerprint — Practical X either group. Furthermore, the results of the ANOVA test Iris — Panasonic Authenticam™ indicated that no statistically significant difference existed Submission X in the starting knowledge of the distance and on-campus Iris Recognition — X students (p = 0.29). This was important to establish, as all Post-Test other comparisons were based on the assumption that the groups started out with similar knowledge. Had the groups not started out with equal levels of similar TEST POPULATION knowledge, then the results of further comparisons would The on-campus IT 345 course had 86 students have been difficult to interpret. enrolled. The course was divided into eight laboratory sections, each of which included between 10 and 12 Module score analysis students. Two teaching assistants taught four sections The total modules score consisted of all students’ each. All on-campus students attended the same lecture post-test, assignment, and practical scores for each period. module, which were then compared by group. The The group of distance students consisted of 27 ANOVA test result indicated that a statistically significant individuals, each of whom had access to a two-hour block difference existed between the distance education and on- of computing time for performing the laboratory campus students. Specifically, the on-campus students experiments. Distance students had access to the had an average score of approximately 87 percent, recording of the lecture that on-campus students received. compared to 80 percent for distance students. This Graduate and undergraduate distance students were resulted in a p value of 0.001. However, the r2 value grouped together as distance students. suggested only 10.10 percent of the difference could be explained by the total modules score alone. Therefore, it RESULTS was desired to determine were the differences were occurring. One-way ANOVA tests were conducted to Module scores were divided into three types, identify and compare various performance measures of parallel to the “modification required” categories denoted both distance education and on-campus students. Each in Table 1. An ANOVA test was run to compare the ANOVA was used to test for statistically significant module scores between the groups when no laboratory differences between the performance of distance activities were required. The results indicate that no education and on-campus students using α =.05. ©2006 WCCSETE March 19 - 22, 2006, São Paulo, BRAZIL World Congress on Computer Science, Engineering and Technology Education 95
  4. 4. statistical difference exists. However, it is interesting to REFERENCES note that on average, the distance students performed better. 1. Moore, M. and G. Kearsley (1996). Distance Education: A Systems View. Belmont, Wadsworth Publishing Company. The next comparison examined the module 2. CDLP (2004). Adult Learning Activities: California Distance scores of the groups when the laboratory activities could Learning Project. Sacramento. 2004. not be modified for distance education. Thus, distance 3. Driscoll, M. (1998). Web Based Training: Using Technology to students answered post-tests without conducting the Design Adult Learning Experiences. San Francisco, Jossey- Bass/Pfeiffer. supporting laboratory activities. The results indicate a 4. The Commonwealth of Learning (2000). The Use of Multimedia in statistically significant difference between the groups’ Distance Education. Vancouver, BC. 2004. module scores. Moreover, the on-campus students 5. Caffarella, R. (2002). Planning Programs for Adult Learners: A averaged approximately six percent higher scores, Practical Guide for Educators, Trainers, and Staff Developers. San Francisco, John Wiley & Sons. resulting in a p value of 0.036. 6. Aktan, B. Bohus, C.A. Crowl, L.A. Shor, M.H. The last of the module breakdowns consists of (1996). Distance learning applied to control engineering the laboratory activities that were modified for distance laboratories. IEEE Transactions on Education. 39(3) p. 320- 326 education. The ANOVA test results indicate that no 7. Leiner, R. (2002). Tele-experiements via Internet a new approach for distance education. 11th Mediterranean Electrotechnical statistical difference exists between the groups. The Conference, IEEE. distance students only scored approximately three percent 8. Hu, J., D. Cordel, et al. (2004). Virtual Laboratory for IT Security lower on average, which is an improvement over module Education. Proceedings of the Conference on Information Systems scores resulting from a lack of modifiable laboratory in E-Business and E-Government (EMISA), Luxembourg. 9. Sickler, N., E. Kukula, et al. (2004). The Development of a activities. However, the resulting p value of 0.056 is Distance Education Class in Automatic Identification and Data borderline with α =.05, which makes it difficult to report a Capture at Purdue University. World Conference on Engineering decision of no difference. and Technology Education, Santos, Brazil. In assessing the overall performance of the students in the course, it is interesting to note that the on- campus and distance students performed as well on their exams. The combined average exam score is the resulting average score of the three exams administered throughout the semester. The ANOVA test result indicates no statistical difference; in fact, the averages are almost identical, resulting in a p value of 0.922. CONCLUSIONS This paper was written to understand whether there were any differences in how and whether the material in an Automatic Identification and Data Capture course is learned by two different populations. Results of the module score analysis indicate that a statistically significant difference exists between the distance education and on-campus students. However, upon module breakdown (whether it had been modified for distance) and further comparison, the results indicate that the difference in overall module scores can be attributed to instances of modules containing laboratory activities that could not be modified for distance education. In the other two instances, where no laboratory exercises were required, and in laboratory exercises with distance modifications, the groups exhibited results that were not statistically different. ©2006 WCCSETE March 19 - 22, 2006, São Paulo, BRAZIL World Congress on Computer Science, Engineering and Technology Education 96