Construction and Evaluation of a Multimedia Mobile Classroom Feedback System Wen-Chen Huang and Shih-Chieh Sung Department of Information Management National Kaohsiung First University of Science and Technology Email:firstname.lastname@example.org AbstractIn the conventional classroom setting, there are many different obstacles to theinteraction between instructor and students, such as limited class hours, fixed seating,and inadequate time for after class meetings. This research develops a multimediamobile classroom feedback system, one which instantly displays any responses (suchas class related questions or the answers to a quiz) on a computer used by theinstructor. This system is employed to reinforce the interaction between instructorsand students and to improve learning efficiency, and the salient features are as follows:1. automated attendance checking mechanism, 2. in-class real-time evaluation, 3.improving students’ willingness to raise questions, 4. instant student grade enquiry, 5.system cross platforms, 6. virtual mobile classroom, 7. data integrity, 8. lowconstruction cost, 9. group learning, and 10. recording the process of learning. Thissystem uses client-server and 3-tier relational database models as the framework.When a class is in progress, the instructor only requires a notebook computer in whicha web browser is installed, while students need a mobile phone or PDA which hasbuilt-in J2ME and wireless network support. A web server and the rear-end databaseare not required to be installed in the vicinity, as long as the network connection isproperly set up. Currently, the system has four different functions: attendancechecking, conducting quizzes, instant question inquiry, and grade inquiry. A survey questionnaire is used for the evaluation of this system (MMCRS).Subjects were two classes of undergraduate students with a sample size of 105. Thequestionnaire considers three perspectives: interactivity, mobility and functionality.The purpose of this survey was to compare the MMCRS and IRS systems and find outtheir differences. The experimental results show that MMCRS is superior to IRS fromall three perspectives.Key words: multimedia, classroom interaction, wireless technique, mobile classroom,M-learning
1. IntroductionExamining the history of the development of Educational Psychology, be itBehaviourism, Constructivism or Cognitivism, the emphasis is always on theimportance of information feedback, which basically has two meanings. The first isthat a teacher should know the learning progress of their students. Throughinformation feedback, they are them able to make appropriate adjustments to theirteaching material or methods. The second one is that the teacher releases informationon the teaching material to the students, who are then to make the necessaryadjustments and improvements to their learning process. In teaching, instructors often adjust their method according to feedback, but withconventional teaching techniques, this feedback is rather poor. It is often difficult for ateacher to monitor the learning progress of all the students due to the large class size.Interaction between students and teachers is thus superficial (e.g. answering questionsby raising hands), impersonal (e.g. answering question together with others) anddelayed (e.g. doing bring-home assignments, having exams and so on). A teacherspends a lot of time on grading students’ work, who then will have to wait at least oneday to know how they perform. Worse still, such delayed feedback may not havemuch impact on students’ learning. Being aware of this problem, teachers haveattempted to make students’ feedback more instantaneous and scientific. However, theresults have not been satisfactory. In most cases, only some students may have thechance to interact with their teacher and their interaction tend to be limited to, forexample, the teacher asking students to raise their hands in agreement, or naming aparticular student to answer a question. Asking questions rarely, a teacher may not beable to have a thorough understanding of students’ learning progress. In other words, ateacher cannot adjust their approach to see to students’ needs and they, too, due to alack of participation, will lose interest in the class. The efficiency of the teaching isthus further lowered, and students may even stop coming to the class. In the conventional classroom setting, various obstacles limit the interactionbetween students and teachers. There are obstacles like limited class hours, fixedseating and inadequate time for after class discussion [1, 2]. So far, most teachingmethods have been concerned with either teaching or learning, and few have focusedon improving classroom interaction. Since the amount of information that is involvedin teaching has been increasing, a corrective method is urgently needed, and with theintroduction of multimedia feedback techniques it is possible to receive instantfeedback from every student in the class. Moreover, such techniques facilitate preciseand scientific data analysis and allow recording of the entire teaching process. Themultimedia mobile classroom feedback system in this paper is developed using aclient-server model. Handheld devices (like a mobile phone or PDA) use a wireless
network (GPRS, 3G or WiFi) to transmit students’ immediate responses, for example,questions that they raise in the class, answers to quizzes (multiple choice, shortanswer, fill-in the blank types of question), and the like. The feedback from studentscan be displayed instantaneously on the teacher’s notebook computer. The teacher canthen use this system with pre-designed materials and then modify the contentaccording to the feedback. In preparing for courses, a teacher can always include thissystem as a piece of auxiliary teaching equipment. Using this tool, the interactionbetween teachers and students is enhanced, improving the efficiency of learning. Tosum up, there are several advantages with this system:1. Automated attendance checking mechanism: It replaces the conventional method by which attendance is checked manually by instructor. With the system, students need to key in their ID number on a handheld device. The teacher will then know immediately which students are absent from the class.2. In-class instant evaluation: In a class, the teacher can give a quiz at any time (multiple choice, short questions and answers, and fill-in the blank types of question) to examine whether students really understand what has been taught. With an instant feedback mechanism like this, students are required to be attentive to the class. In addition, the teacher is able to understand and monitor the effect of their teaching on the students.3. Improving the willingness to ask questions: It is often the case that students may wish to ask questions, but they are unable to do so due to shyness or thinking that their question is not appropriate. By using this system, questions can be asked upon without registering who has made them. In so doing, the teacher becomes aware of the problems in students’ learning, and thus able to devise ways to assist them and raise the quality of teaching.4. Instant results checking: Students are able to check their own results at any time through a handheld device, with which their privacy is better protected. With such an arrangement, it is also easier for teacher to publish students’ results.5. Multi-platform system: The construction of this system does not neeed to be tied to any particular platform, as Java is used. All that a student needs is a handheld device which supports J2ME. Server-side implementation can either be with Microsoft or Unix-like platforms.6. Virtual Mobile Classroom: Since a wireless network is used as the transmitting media, students are able to use this system whenever they like (be it in or out of the class). When students are reviewing their lessons without a computer, they can still use this system to fire question to their instructor.7. Comprehensiveness of the collected data: This system employs a database to store student-related information. Through the use of the transaction and locking
mechanisms of database, asking questions and sending answers simultaneously will not cause any problem, and thus data is properly collected.8. Low construction cost: The client-side equipment, i.e. mobile devices that students will use, do not need to be redesigned. It can be a mobile phone, a PDA or a notebook computer, something which students may already possess. As to the server-side machine, it requires only a web-server.9. Group learning: Very often, a group discussion in class only allows students to present the result of their discussion. It may not be easy to tell whether all of the students have really learned something from the person who is presenting the work. In this case, an instructor can pass the system to students who are presenting ther work and they can set a quiz for other students or receive questions from them, while the instructor can make observations without interrupting the students.10. Recording learning process: The discussion and interaction between the teacher and students can be stored in the database which is in the back-end of the system. By reproducing learning process, it allows the teacher to investigate any problems that might hinder students’ learning. Consequently, in preparing the teaching materials, adjustments can be made for future classes.These are some of the advantages of this system. In the next section, a literaturereview is included which also stresses the importance of in-class interaction, and theinadequacy of feedback in the conventional classroom setting. In section three, thetheoretical framework of this system is discussed while GPRS and pressure test of thesystem are shown in section four. The summary section discusses the application andfuture development of the system.2. Literature ReviewIn-class interaction is very important in teaching, and one of the key tools forsuccessful learning.[1, 3,4, 5, 6, 7]. In order to encourage students to be moreinvolved in their learning, it is essential for an instructor to use properly preparedteaching materials to raise the interest of the class, and to cultivate their sense ofinvolvement by increasing interaction with them. The more interaction, the greater themotivation of learning will be. The same is also true for students’ concentration onand sense of involvement with the class. Hence, the concept of in-class interaction isrelated to what three different educational theories, Behaviourism, Constructivism andCognitivism, have suggested. [7,8]2.1 In-Class InteractionThe definition of ‘in-class interaction’ varies, but can be basically divided into five
categories. The first one is ‘active involvement’: in-class interaction is seen as anactivity that a learner can actively participate in. Through his or her involvement, anactive learning environment can be constructed. The second category concerns‘communication’: in-class interaction is regarded as a way for learners and educatorsto communicate with one other in a one-to-many form. In other words, the interactionbetween teachers and students is done through communication. An educator tries tounderstand the needs of learners, while learners try to make sense of the knowledgeoffered by the educator. Similarly, the third category focuses on ‘instructor-learnercommunication’. However, the in-class interaction is based on the communicationbetween one learner and one educator. It is a one-to-one interaction model. A teachertries to understand the needs and problems of one student at a time. Before a teachingrelated interactive model can be developed, the educator and the learner shouldcommunicate with each. The fourth category sees interaction as ‘collaborative’:in-class interaction in this definition is behaviour of cooperation and social function.The relationship between students and teachers is some form of cooperation, and so isthe relationship among students. The teacher and their group of students are just likemembers of a society, and by being cooperative they can encourage and help oneanother. The fifth category is more concerned with ‘instructional activities andtechnologies’: here in-class interaction is assigned to the field of teaching activity andtechnology.2.2 Mobile NetworkingBen Mousaa  lists some advantages of mobile networking. Firstly, in general, theapplication software of mobile networking allows users to control and filter theinformation on mobile devices, and can also be used as a means of communication.Mobile devices often have two distinct characteristics, in that they are personalisedand individualised. Secondly, mobile networking is able to improve collaborationamong people. Through the rapid connection of mobile networking and instant datatransmission services, users can interact with one other without any limitations interms of time and place. Users can decide what the most appropriate strategy is forthemselves. Lastly, mobile networking is customer-oriented. When a user tries tostrike a balance between learning, living and work, it is necessary to optimise the timeavailable. As mobile network can be accessed from anywhere and at any time, andusers can thus make good use of their time with ease. These three advantages facilitatea better learning environment.2.3 Digital Learning EnvironmentSharples  emphasises that learning and technology development cannot be
separated, as learning has become more personalised and learner-centred, morecollaborative in nature and unrestricted in time and place. Therefore, Sharples proposes a strategic framework, focusing on applying technology to the learningenvironment. This framework consists of five different methods: 1. an intelligenttutoring system replaces the tutor in a conventional classroom setting. However, asuccessful working model for this method is yet to be constructed, as replacing a realtutor with an intelligent tutoring system is not easy. For example, a system may notpossess all the knowledge needed on a specific field. 2. Software or tool programmesare instead used to play the role of pedagogical agents which give suggestions tostudents. 3. System tools and resources are used to help the student in learning andorganising new knowledge. 4. Personalised communication aids are used to show theresult of learning, taking into consideration the ability of the learner. 5. A simulatedclassroom allows teachers and students to continuously interact with one other wellafter the real class has ended. Siau et al..  suggest that a classroom feedback system does increase theinteraction between teachers and students. However, there are several disadvantageswith the existing systems: 1. some of the functions in the student’s remote controllerdo not work properly. For example, when some remote controllers are used together,answers from students do not get received properly by the system, or the remotecontroller is out of the range of infra-red receiver. 2. Quizzes provided by theclassroom feedback system are confined to multiple choice and true-or-false questions.3. Some students tend to ignore the in-class feedback system. 4. Students may fiddlewith the remote controller and be distracted from the class. 5. When the classroomfeedback system is used, it may take away some of the lecturing time. Markett, etc.  use Short Message Service (SMS) in mobile phones to increaseinteraction between teachers and students, as the latter’s questions can be transmittedby using SMS messages. However, there are problems with such an arrangement. Oneis students can only use mobile phone for SMS, not PDAs or notebook computers.Also, the number of characters that can be sent using mobile phone is limited. If amessage is long, it will be broken up and delivered in several messages which may bedifficult to read. Moreover, students may use SMS for things other than class relatedpurposes. There is also the possibility of repeatedly sending the same SMS message.In such a case, the large number of repeated SMS messages could be a burden to theteacher. Finally, as the average price of sending a text message in Taiwan is about 3 to5 Taiwan dollars, if the amount of text messages to be sent is high then students willhave to pay a relatively large sum of money. According to the earlier studies on in-class interaction, when introducing aspecific handheld device for students to participate in an activity, the rate of usage of
the system drops, decreasing about 25 to 30%. As the screen of a mobile phone israther small, it will not become an eye-sore to students and teacher . Also, thewireless networks on mobile devices (GPRS, Wi-Fi or 3G) are operated andmaintained by independent companies, so students will not need to do much to use thedevice. Considering the disadvantages and platforms of various mobile devices, thissystem is designed so that users are also able to use it on a PDA or notebookcomputer.3. MethodologyInformation Communication Technology (ICT) has been proved to be able to enhancelearning, especially when it is combined with learner-centred teaching method. Theone-to-one digital classroom concept proposed by Liang et al.  can be integratedwith information and telecommunication technologies, as seen in Fig. 1. Thefollowing is a traditional classroom setting. Fig.1 Digitalised Classroom Setting3.1 Mobile Interaction Framework in a Conventional ClassroomThus framework includes three levels. The first is about in-class interaction, the
second presents the advantages of mobile telecommunication, and the third is thee-learning environment, as shown in Fig. 2. Fig. 2. Mobile Interaction Framework in Conventional ClassroomThe multimedia mobile classroom feedback system has four different functions,which are shown in green. They are Roll Call, Quiz, Instant Q&A and Grade Query.The correspondent behaviour in the classroom will be as follows: 1. making studentsactively participate in the class: Before class is started, the teacher often checks theattendance of students. Therefore, the sense of participation of students starts withchecking themselves into the class, with logging themselves into the system. Afterregistering their attendance, students are able to use the system to ask or answerquestions as they wish. 2. Providing one-to-many interaction between the teacher andstudents: students and teacher should begin to interact instantly to maintain the senseof participation. By conducting an instant quiz, the teacher is able to have some ideaof whether students really understand what has been taught. Since information is sentinstantly and anonymously, students are able to freely pick answers withoutembarrassment, while the teacher is able to adjust the class according to the responses.3. Facilitating one-to-one interaction between teacher and student: if a teacher does
not know his or her students’ learning problems, can only answer a small number ofstudents queries, or if some of the students are shy or scared of asking questionsdirectly, their learning will be impeded. Through this system, students can askquestions instantly and anonymously. In this way, the teacher is aware of everystudents problem and can address their questions right away. This will heightenstudents’ sense of participation in the class, increase their interaction with the teacherand other students, and finally improve their learning. 4. Applying technologies toclass activities. The activities include attendance taking, assigning quizzes, askingquestions and grade enquiries, which can all be done through students mobile devices.The centre of the graph, which is in purple, is the result of a series of in-classactivities. Teachers and students can use this system to propose and discuss questionsto enhance in-class interaction. The part in orange is about the advantages of mobilenetworking. In-class activities and mobile networking can be combined to get a betterresult. When a student asks or answers questions, it is transmitted anonymously, andthus they can do it on their own without any interference from others. As theinformation received by teacher in class arrives instantly, explanations can be madestraight away. Lastly, the blue part is the strategies used in mobile classroominteraction. In a class, a tutor is the main actor while mobile devices are the subsidiarysystem tools, but interaction between students and the teacher can continue after class,in a virtual mobile classroom. As the attitude toward learning and the results ofstudents are positively affected by improved in-class interaction, this framework isable to support the concepts proposed in the three educational psychology theoriesdescribed previously, Behaviourism, Constructivism and Cognitivism.3.2 System FrameworkThis system utilises a client-server model. On the client-side, it is a student instantfeedback system, while the tutor’s class management system resides in a server whichuses a three-tier relational database. The student instant feedback system includes fourdifferent modules, roll call, quiz, grade inquiry and question asking. The tutor’s classmanagement system corresponds with the three modules in the front, which are rollcall switch, quiz switch and score inputting. Through the tutor’s instant feedbackinterface, one is able to find all the feedback information from students. The system isshown in Fig. 3.
Fig. 3. A Framework of the Multimedia Mobile Classroom Feedback System The Presentation Tier of the client site uses a hand-held device, such as a mobilephone or PDA. The application software of the Presentation Tier is developed byusing Sun J2ME, while the server site of the Logic Tier is programmed by PHP.MySQL is used for the relational database system in the Data Tier. This is shown inFig. 4.
Fig. 4 Relational Database FrameworkFig. 5 shows the user interface used on a mobile phone. It displays the overallfunctions, which are the roll call, multiple-choice quiz, text quiz, text question asking,voice question asking, grade query, change password, and logout. Fig. 5 The user interface of a handheld device4 Experimental Results The survey questionnaire is used for the evaluation of this system (MMCRS), andthe subjects were two classes of undergraduate students with a sample size of 105.The procedure of the data collection was as follows. At first, subjects used the IRSsystem  then the MMCRS system, each for one week, and then the questionnairewas conducted. The questionnaire consisted of three perspectives: interactivity,mobility and functionality. The purpose of this survey was to compare the above twosystems and find out their differences. The results and analysis are as follows:(1) Interactivity The purpose of the MMCRS and IRS systems is to enhance classroom
interactivity, and the questionnaire results on this are are listed in Table 1. Table 1: Results of interactivity questionnaire (n = 105) Mean Std. dev. I feel that MMCRS is better than IRS for teacher-student interaction in 3.88 .829 classroom. I feel that MMCRS is better than IRS for discussion in classroom. 3.82 .907 I feel that MMCRS is better than IRS for concentration in studying. 3.82 .896 I feel that MMCRS is better than IRS for understanding the teaching 3.77 .933 material. I feel that MMCRS is better than IRS for knowing whether students 3.87 .878 understanding the teaching material. I feel that MMCRS is better than IRS for knowing whether students’ are 3.90 .887 keeping up with the teaching material. Overall, I prefer MMCRS to IRS for student-teacher interaction. 4.03 .871 5 = strongly agree, 1 = strongly disagree Table 1 shows that there is no significant difference between MMCRS and IRS for student-teacher interaction. Both systems improve interaction in the classroom, to both aid students’ better understand the teaching material and allow the teachers to understand and monitor the students’ progress. However, most students prefer MMCRS to IRS.(2) Mobility The tool used for MMCRS is mobile devices, and thus the advantages of mobile communication – individualised, user-oriented, and coordinated – also apply to MMCRS. The questionnaire’s results for mobility are listed in Table 2. Table 2: Mobility Survey Results (n = 105) Mean Std. dev. I feel that MMCRS is better than IRS for accessing the internet at anytime 4.20 .671 and anywhere. I feel that MMCRS is better than IRS for providing personal information, 4.18 .647 such as grades. I feel that MMCRS is better than IRS for effectively using my time to 4.08 .661 study, such as asking an question while studying in the library. I feel that MMCRS is better than IRS for engaging me to discuss the class 3.99 .946 contents with classmates. I feel that MMCRS is better than IRS for understanding my the messages 4.08 .756 I transfer. Overall, I feel that MMCRS is better than IRS ass it is is easier to carry 4.36 .735
and use. 5 = strongly agree, 1 = strongly disagree Table 2 shows that the advantages of mobile communication greatly add to the appeal of MMCRS, and students think that it enhances classroom learning and increases discussion between students and the teacher. Overall, students also believe that MMCRS is better than IRS as it is both easier to carry and use.(3) Functionality This perspective is compares the functionality between MMCRS and IRS. The purpose of this part of the questionnaire is to understand which parts of MMCRS need further improvement, and the results are listed in Table 3. Table 3: The results for functionality (n = 105) Mean Std. dev. I feel that MMCRS is better than IRS for the ‘roll call’ function. 4.10 .827 I feel that MMCRS is better than IRS for the ‘instant quiz’ function. 4.00 .820 I feel that MMCRS is better than IRS for the ‘instant questioning’ 4.05 .752 function. I can ask a question via voice at anytime and anyplace by using 4.09 .786 MMCRS. I feel that it is easy to use voice functionality to ask a question by using 3.95 .881 MMCRS. I can inquiry about my score at anytime and anywhere by using 4.15 .632 MMCRS. It is easy to use the score inquiry function with MMCRS. 4.08 .661 It is convenient to use MMCRS on any platform. 4.29 .675 Overall, I agree that MMCRS is superior to IRS for functionality. 4.18 .852 5 = strongly agree, 1 = strongly disagree Table 3 shows that students are convinced of the superiority of MMCRS over IRS from a functional perspective. The unique functions of voice questioning and score inquiring for MMCRS are easy to use and useful on any platform.5. Conclusions This research uses the students’ most commonly used handhold devices, themobile phone, PDA, notebook computer, and so on, and selects an anonymousmethod to let students fully expression their opinions and thus obtains more
interactions in the classroom with the teacher. Allowing the student to initiate the sinteraction increases participation in the classroom, builds driving-type in the campusthe learning environment, leading to better teaching quality and the achievement ofmore study goals. However, despite the positive findings above, it should still be noted that theeffect of MMCRS depends on the students’ learning attitudes. For the instantquestioning model, if students ask some unrelated questions it is unlikely to lead tolearning. However, teachers who used MMCRS may select some key point questionsfor use in the classroom, and in this way the system may enhance students’concentration and lead them to pay more attention to the important issues in thecourse. There are long-term and short-term directions for future research, For theshort-term, the response time of the voice question-asking mechanism should beimproved. For the long-term direction, it would be helpful to include a videoconference function and a homework deadline reminder.References T. Liu, J. Liang, H. Wang, T. Chan, and L. Wei, “Embedding educlick in classroom to enhance interaction,” in Proc. Int. Conf. Computers in Education (ICCE)., Hong Kong, China, 2003, pp. 117–125. S. W. Draper and M. I. Brown, “Increasing interactivity in lectures using an electronic voting system,” J. Comput. Assist. Learn., vol. 20, 2004, pp. 81–94. C. P. Fulford and S. Zhang, “Perceptions of interaction: The critical predictor in distance education,” Amer. J. Distance Educ., vol. 7, no. 3, 1993, pp. 8–21. C. Chou, “Interactivity and interactive functions in web-based learning systems: A technical framework for designers,” Br. J. Educ. Technol., vol. 34, no. 3, 2003, pp. 265–279. B. Bannan-Ritland, “Computer-mediated communication, elearning, and interactivity: A review of the research,” Quart. Rev. Distance Educ., vol. 3, no. 2, 2002, pp. 161–179. M. C. Wang, G. D. Haertel, and H. J. Walberg, “What influences learning? A content analysis of review literature,” J. Educ. Res., vol. 84, no. 1, 1992, pp. 30–43. R. Sims, “Promises of interactivity: Aligning learner perceptions and expectations with strategies for flexible and online learning,” Distance Educ., vol. 24, no. 1, 2003, pp. 87–103. W. D. Haseman, V. N. Polatoglu, and K. Ramamurthy, “An empirical investigation
of the influences of the degree of interactivity on user outcomes in a multimedia environment,” Inform. Resources Manage. J., vol. 15, no. 2, 2002, pp. 31–48. C. BenMoussa, “Workers on the move: new opportunities through mobile commerce.” Presented at the Stockholm Mobility Roundtable, May 2003, pp. 22–23. M. Sharples, “The design of personal mobile technologies for lifelong learning.” Computers and Education, 34, 2000, pp. 177–193. Keng Siau, Hong Sheng, Nah, F.F.-Hoon, “Use of a classroom response system to enhance classroom interactivity,” Education, IEEE Transactions on Volume 49, Issue 3, Aug. 2006, pp.398 – 403. C. Markett, I. Arnedillo SAAnchez, S. Weber, B. Tangney, "Using short message service to encourage interactivity in the classroom," Computers & Education Volume: 46, Issue: 3, April 2006, pp. 280-293. S. W. Draper, and M. I. Brown, “Increasing interactivity in lectures using an electronic voting system.” Journal of Computer Assisted Learning, 20, April 2006, pp. 81–94. K. Nyiri, “Mobile communication: essays on cognition and community.” Vienna: Passagen Verlag, 2003. Zhu & Kaplan. “Technology and Teaching.” In W. J. McKeachie (Ed.), McKeachie’s teaching tips: strategies, research, and theory for college and university teachers, Boston: Houghton Mifflin Company, 2002, pp. 204-224. J.K. Liang, T.C. Liu, H.Y. Wang, B. Chang, Y.C. Deng, J.C. Yang, C.Y. Chou, H.W. Ko, S. Yang, and T.W. Chan, "A few design perspectives on one-on-one digital classroom environment," Journal of Computer Assisted Learning, Volume 21 Issue 3, June, 2005, pp. 159-237. F. Nah, K. Siau, and H. Sheng, “The value of mobile applications: A utility company study,” Commun. ACM, vol. 48, no. 2, 2005, pp. 85–90. H. Sheng, F. Nah, and K. Siau, “Strategic implications of mobile technology: A case study using value-focused thinking,” J. Strategic Inf. Syst., vol. 14, no. 3, 2005, pp. 269–290.