Final year design project report - Studies in application of augmented reality in E Learning Courses

4,232 views
4,150 views

Published on

Published in: Education, Technology
1 Comment
6 Likes
Statistics
Notes
No Downloads
Views
Total views
4,232
On SlideShare
0
From Embeds
0
Number of Embeds
360
Actions
Shares
0
Downloads
0
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide

Final year design project report - Studies in application of augmented reality in E Learning Courses

  1. 1. Final Year Design Project: Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati Indian Institute of Technology Guwahati Studies in application of augmented reality in E Learning Courses Himanshu Bansal (516) & Mannu Amrit (523) Final Year Design Project (2013 – 2014) Project Guide: Prof. (Dr). Pradeep Yammiyavar Head, Center for Educational Technology, IIT Guwahati Department of Design, IIT Guwahati ////////////////////////////////////////////////////////////////////////////
  2. 2. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 3 Acronyms Used AR- Augmented Reality NCERT- National Council of Educational Research and Training GUI - Graphical User Interface 3D - 3 Dimensional CCP- Cubic Closed Packing HCP- Hexagonal Closed Packing FCC- Face Centered Cubic OV – Octahedral Void TV – Tetrahedral Void VARK – Visual Auditory Reading & Kinesthetic PSVT:R – Purdue Spatial Visualization Test: Rotation PEOU – Perceived ease of use PU – Perceived Usefulness AT – Attitude BI – Behavioral Intention SA – Self Efficacy PE – Perceived Enjoyment
  3. 3. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 4 Figures & Images Used Figure 1: Chemistry + Augmented Reality + E Learning Figure 2: Homepage, www.coursera.org Figure 3: The Johnstone triangle Figure 4: Connecting Design Project 3 and Design Project 4 Figure 5: The Johnstone triangle Figure 6: 3D structure, tetragonal voids, Page 17, Standard XII NCERT Figure 7: Taxonomy of mixed reality including real to virtual environments Figure 8: An AR system and the physical model [6] Figure 9: NCERT Chemistry Textbook, Standard XII Figure 10: Dependent Variables Figure 11: Independent Variables Figure 12: Interview at Oriental Tutorials, Guwahati Figure 13: Interview at Kendriya Vidyalaya, IIT Guwahati Figure 14, 15: D Fusion Studio Figure 16: Vuforia by Qualcomm Figure 17: Unity software Figure 18: Sketchup software Figure 19: Virtual Buttons (in blue) and GUI buttons (in black) Figure 20: Task Flow Diagram, Module 1 Figure 21: App Screenshots, Module 1 Figure 22: App Screenshots, Module 1 Figure 23: AppTest Screenshot, Module 1 Figure 24: Task Flow Diagram, Module 2 Figure 25: App Screenshots, Module 2 Figure 26: 3D Models, Module 1 Figure 27: 3D Models, Module 1 and 2 Figure 28: 3D Models, Module 2 Figure 29: Students using the prototype
  4. 4. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 5 Figure 30: Classification of A.R publications by evaluation method / approach Figure 31: Technology Acceptance Model Figure 32: Sample PSVT Question Figure 33: Octahedral void as seen in new prototype Figure 34: Removal of virtual buttons and changes in GUI Figure 35: Addition of interactivity by touch Figure 36: Zoomed in view of prototype Figure 37: PSVT A.R Prototype Figure 38: Web Interface (Voids) Figure 39: Web Interface (PSVT) Figure 40: Participants filling Pre Questionnaire Figure 41: Experiment with A.R (up) & Web based system (below) Figure 42: Experiment with A.R (up) & Web based system (below) Figure 43: Pre Questionnaire mean & standard deviation Figure 44: Pre Questionnaire VARK mean & standard deviation Figure 45: TAM Mean & Standard Deviation Figure 46: Spearman Rho Corelation value table for Augmented Reality users Figure 47: Spearman Rho Corelation value table for Web users
  5. 5. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 6 Contents Acknowledgment 1 IPR Declaration 2 Acronyms 3 Figures & Images used 4 Chapter 1 – Introduction 1.1 Abstract 8 1.2 Motivation 10 1.3 Objectives 11 Chapter 2 - Literature Review 2.1 Why Chemistry? 12 2.2 Augmented Reality 13 2.3 Existing Work 15 Chapter 3 - Project Timeline 18 Chapter 4 - Design Methodology 4.1 Case Study Topic 19 4.2 Research Design 20 4.3 Design Guidelines 21 4.4 User Requirement 22 Analysis 4.4.1 Interview 23 Questionnaire 4.4.2 Summary of 24 Responses 4.4.3 Insights from 24 Interviews Chapter 5 - Development 5.1 D Fusion 27 5.2 Vuforia & Unity basics 28 5.3 Virtual Button & GUI 29 5.4 Application 30 5.5 App Flow 30 5.5.1 Module 1 30 5.5.2 Module 2 33 5.6 Audio Components 35 Chapter 6 – Initial Feedback 38 Chapter 7 – Literature Review (Phase II) 33 7.1 Evaluation techniques 39 7.2 Spatial Ability 40 7.3 Technology Acceptance 41 Model 7.4 PSVT 42 7.5 VARK 43 Chapter 8 Improvisations in A.R prototype 8.1 Introduction 44 8.2 Changes 45
  6. 6. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 7 8.3 Final GUI Walkthrough 47 8.4 Additional Application 48 Chapter 9 – Web Interface 9.1 Introduction 49 9.2 Design 50 9.3 Development 51 Chapter 10 – Research Methodology 10.1 Aims & Rationale 52 10.2 Experiment Design 53 10.3 Research Questions 54 10.4 Participants 55 10.5 Setup & materials 56 10.6 Procedure 57 Chapter 11 – Results 11.1 Quantitative 59 11.2 Qualitative 65 Chapter 12 – Discussion 73 Chapter 13 – Proposed design 86 guidelines Chapter 14 – Conclusion 88 Chapter 15 - References 90 Appendix Summary of Responses Image Tracker The VARK Questionnaire Solid States Questionnaire- Pre Questionnaire Solid States Questionnaire- Main Study The Purdue Visualization of Rotations Test Technology Acceptance Model Questionnaire Web Quantitative Data (Part 1) Web Quantitative Data (Part 2) AR Quantitative Data (Part 1) AR Quantitative Data (Part 2)
  7. 7. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 8 Figure1: Chemistry + Augmented Reality + E Learning Chapter 1: Introduction 1.1 Abstract Previous Studies have indicated that specific concepts in chemistry education require visuospatial skills by students. Researchers have explored augmented reality (AR) in aiding the spatial visualization needs of the students in subjects like Astronomy & Geometry. Augmented reality is a popular technology which has come into the limelight in the recent years. In layman terms, it is a technology which combines real and virtual imagery at the same time. It is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video and graphics. Being very interactive in real time, its implications and use cases have evolved into different domains: health, education, entertainment etc. The domain for application of this technology of particular to interest for us in this project is E Learning. E Learning refers to training initiatives which provide learning material, course communications, and the delivery of course content electronically through technology mediation. In this project, both the domains of AR reality and E Learning have been explored in the context of Chemistry for high school students. The project was planned out such that the first phase (Design Project III) began with a qualitative study conducted with five high school chemistry teachers in India. This study was conducted with the aim to identify existing pedagogical patterns and issues related to Solid State Chemistry taught in senior high schools in India. The results of this study were analyzed and were found to be validating the existing literature in chemistry education. Based on inferences from this study combined with principles proposed in previous research, we then conceptualized and developed an AR based android application for mobile and tablet devices. This application uses
  8. 8. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 9 standard XII NCERT textbook images as markers/reference to augment dynamic 3 dimensional content. The content of the application, decided on the basis of inputs from study, is interactive and supported with animation and audio based feedback. The next phase (Design Project IV) focused on testing this application through a comparative analysis with existing e-learning modalities such as web based. The aim of this experiment was twofold. The first was to understand and establish if an A.R based e-learning tool would actually be helpful to students in content learning, 3D spatial visualization and behavioral intention of users towards the system. The second aim was to identify its strengths when compared to current e learning modalities and finally identify its shortcomings and weaknesses. Based on the quantitative analysis of results of our experiment as well as qualitative feedback received from participants during the experiment, we establish how A.R based tools have immense potential as self-sufficient learning modules and propose design inferences to be considered while designing AR based solutions.
  9. 9. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 10 Figure 2: Homepage, www.coursera.org 1.2 Motivation Solid State Chemistry which is taught as the first topic in standard XII in high school chemistry in India involves several concepts with 3 dimensional visualization of atoms and molecules. Having faced difficulties ourselves in this domain in our school days, we explored it further as our topic for addressing an augmented reality based solution. Also, in parallel, with websites such as Coursera, EdX and Udacity gaining immense popularity amongst students in the recent few years, we believe that E Learning is an area wherein lies immense potential for innovation. The current model of teaching in E Learning lies heavily on video lectures, which is a passive means of interaction. Thus, we worked towards the development of an AR based tool and an experiment to test it versus conventional teaching practices which could potentially throw insights on its feasibility, interactivity, user engagement and effectiveness in learning in the future.
  10. 10. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 11 Figure 4: Connecting Design Project 3 and Design Project 4 1.3 Objectives The key objectives for the project were identified as:  Identify scope of Augmented Reality in E Learning and in our subject of interest - Solid State Chemistry.  Conduct user study for qualitative feedback about teaching methodologies for Chemistry concepts as well as the existing E Learning model.  Develop an AR based E Learning solution for a specific section in Solid State Chemistry.  Conduct a comparative study of the developed solution with a conventional e learning solution available as of today.  Identify strengths and weaknesses of A.R based E Learning tools and propose design guidelines for such systems.
  11. 11. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 12 Figure 5: The Johnstone triangle Figure 6: 3D structure, tetragonal voids, Page 17, Standard XII NCERT Chapter 2: Literature Review 2.1 Why Chemistry? One of the challenges of chemistry education is that it must address multiple levels of representation, from the macro level (tangible and observable) to the sub-micro explanatory level (atoms, molecules, ions) [Johnstone,2010]. For novices, understanding these multiple levels and the relationships among them can be challenging. Digital technology, which offers numerous ways to represent information, has come to play an important role in chemistry education, but there are key aspects of interaction and interoperability (i.e. differing operating systems) that still present problems. Modern chemistry is characterized by interdependent, networked thinking in different representational domains. This consideration is in the core of Johnstone’s (1991) famous contribution: ‘Why is science difficult to learn? Johnstone explained that learning and thinking in modern chemistry always take place in a constant shift between three different representational domains: the macroscopic, sub-microscopic, and symbolic domain. If these three domains (including the accompanying levels between the macroscopic and sub- microscopic domains) and their interactions are misinterpreted, scientifically unreliable interpretations will necessarily emerge as a result [Johnstone, 1991].
  12. 12. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 13 Figure 7: Taxonomy of mixed reality including real to virtual environments 2.2 Augmented Reality Augmented Reality (AR) is a technology that allows virtual images to be seamlessly mixed with the real world [Bauer et.al. 2001, Hampshire et.al. 2006, Steed et.al. 1996]. AR stands between virtual reality and the real environment. In contrast, Augmented Virtuality is a technology that enhances the users’ reality by inserting a real object into a virtual environment. AR and a virtual environment can be divided depending on whether the environment or object in the real world appears or not. Hence, an AR application requires a video input device, e.g. a video camera, to receive an input from the real world, and it should also be made meticulously so that the user cannot distinguish the virtual world from the real world. In addition, AR has real-time properties, since the user should be able to watch the screen. As the screen with the AR is displayed to the user, the user experiences a higher level of immersion with AR as compared to other technologies. Augmented reality technology has been used in several fields [Azuma, 1997] as varied as medicine, robotics, manufacturing, machine repair, aircraft simulations, entertainment and gaming [Oda et.al. 2008]. This research presented concentrates on the use of augmented reality in education, more specifically E- Learning. Several authors [Pantelidis 1995, Winn, 1993] suggested that virtual reality increases motivation, contributes to better learning, and enhances the educational experience for students. Although AR applications for education have been in place, its impact on learning has only now begun to be explored. AR is a medium which overlays virtual objects on the real world. What features does AR have to help conceptual learning? As a new technology, firstly, AR naturally draws people’s attention.
  13. 13. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 14 Drawing students’ attention is an important factor in instruction [Gagne et.al. 1992]. Second, it is a trend to use technology to create a constructivist environment to enhance learning [Dede. 1995]. AR offers an alternative way to see the chemistry world and allows students to interact with the system and discover knowledge by themselves. Thirdly, AR not only creates visual images, but also conveys the spatial cues directly to users [Shelton et.al. 2004]. In other words, by using AR users can obtain a sense of spatial feeling. AR has great potential to be applied to the knowledge domain of spatial concepts. Another feature of AR that enhances learning is that AR allows users to interact with the system by using their body, especially the hands, and provides “sensorimotor feedback” [Shelton et.al. 2004]. The direct manipulation of AR can supplement the deficiency of mouse- based computer-generated visualization since mouse manipulation is an indirect physical manipulation [Shelton et.al. 2004]. Lastly, AR can be a tool which requires users to interact and think carefully [Schank et.al. 2002]. Since users have to concentrate on the AR system and focus on the virtual objects, they may pay more attention to think about what happens next, and thus make them think more deliberately. Overall, AR as an educational medium provides a great alternative environment for students to learn abstract concepts.
  14. 14. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 15 Figure 8: An AR system and the physical model [6] 2.3 Existing Work A lot of research has been done towards application of Augmented Reality in education. Studierstube was one of the initial projects in this direction. In [Szalavári et. al., 1998; Schmalstieg et. al., 2002], researchers have presented collaborative, multi-user augmented reality system Studierstube in which users wear lightweight see-through head mounted displays to access three- dimensional stereoscopic graphics. Initially, collaborative augmented reality with personal Interaction Panel, a two- handed interface system was implemented which was later extended to heterogeneous distributed architecture to become useful in multiple ways. MagicBook [Billinghurst et. al., 2001] is a project in which digital 3d models are embedded onto real book pages. It’s Initial user feedback was quite positive and even complete novices felt that they could use the interface and became part of the virtual scenes. Construct3D is a three dimensional geometric construction tool based on the collaborative augmented reality system ‘Studierstube’ which is specifically designed for mathematics and geometry education [Kaufmann et. al., 2000 & 2003 ]. Later on, its researchers went on evaluate the system in terms of usability [Kaufman & Dünser, 2007] and its potential to train spatial of the students [Dünser, Steinbügl et. al., 2006]. They have reported that augmented reality can be used to develop useful tools for spatial ability training. But traditional spatial ability measures probably do not cover all skills that are used when working in 3-D space. Thus new tools to measure spatial ability directly in 3-D would be desirable.In usability evaluation study they found out that usability of Construct3D was rated higher than the usability of a desktop based geometry education application. This may be due to the more intuitive workflow when working on 3D tasks.
  15. 15. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 16 [Martín-Gutiérrez & Luís Saorín et. al., 2010] presented an application of augmented reality for improving spatial abilities of engineering students. An augmented book called AR-Dehaes has been designed to provide 3D virtual models that help students to perform visualization tasks to promote the development of their spatial ability during a short remedial course. In their next paper [Martín-Gutiérrez & Contero et. al., 2010], researchers evaluated its potential and usability. They suggested AR-Dehaes as an efficient and effective material for developing spatial abilities and for learning engineering graphics contents. In the usability assessment, AR- Dehaes was scored very positively by students with regard to both the teaching material and the technology used. ARIES [Wojciechowski and Cellary, 2013] is a very recent project towards implementing augmented reality in education in which learners’ attitude towards the system was evaluated using Technology Acceptance Model. A recent study [Chen, 2006] investigated how chemistry students interacted with augmented reality and physical models and evaluated the student perceptions regarding these two representations in learning about amino acids. Although there were students who liked using AR to learn about the amino acids because it was portable and easy to make as well as it allowed the students to observe the structures in more detail others felt uncomfortable using the AR marker because it wouldn’t work if the student flipped the marker since it works on marker recognition. The study suggests that using a cube to convey the AR recognition pattern might be a solution to addressing the issue associated with flipping the marker. This research provides guidelines concerning designing the AR environment for a classroom setting [Chen, 2006]. The application shown in Figure 8 includes both an AR marker and a physical model, which are placed on the desk side by side. They showed ball-and-stick models of the acids. Participants could choose from the
  16. 16. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 17 AR marker or the physical model to learn about the acids. One paper [Chen, 2006] compares the use of AR marker and a physical model to see which one is more effective in helping students learn about the acids.
  17. 17. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 18 Week Dates Work 1, 2 Aug 19th - Sep 1st Literature Study + Analysis 3 Sep 2nd - Sep 8th Need Finding, How our project is unique 4 Sept 9th - Sept 15th Testing with D Fusion Studio 5 Sept 16th - Sept 22nd Report, PPT 6 Sept 23rd - Sept 29th Mid Sem Week + User Research 7 Sept 30th - Oct 6th Getting started with building AR interfaces 10-Aug Oct 7th - Oct 27th Development 11 Oct 28th - Nov 3rd Debugging 12 Nov 4th - Nov 10th Finishing Touches 13 Nov 11th - Nov 17th User Testing, Report Submission 14 Nov 18th - Nov 24th Presentation, Winding Up Chapter 3 Project Timeline The project has been divided into two phases: Phase 1 – Design Project III August 2013 – November 2013 This phase would primarily focus on development of the AR tool based on identified content through research. Phase 2 – Design Project IV January 2014 – April 2014 This phase would focus on testing the developed product in an experiment against existing teaching modalities. This would be followed by drawing inferences from the experiment and arriving at a conclusion about the use of augmented reality in E learning. Month Work January Literature review for evaluation techniques Prepare publication for submission February Finalize design for comparative analysis Develop Web Interface March Questionnaire Design + Pre pilot Comparative analysis - (Phase 1 + Phase II) April Analysis of results Thesis report Final Exhibition
  18. 18. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 19 Figure 9: NCERT Chemistry Textbook, Standard XII Chapter 4 Methodology 4.1 Case Study Topic To study the application of Augmented Reality in E-Learning courses, we chose Solid States, first chapter in Chemistry book of class 12th according to NCERT course curriculum as our case study topic. This chapter mostly deals with 3d arrangement of atoms of crystalline metallic, non-metallic elements and ionic and covalent compounds which need the students to understand the concepts sub-micro and symbolic level at the same time. More importantly, it requires students to visualize the atomic arrangement in 3d space which deals with Visio-spatial thinking capability of the students.
  19. 19. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 20 Figure 10: Dependent Variables Figure 11: Independent Variables 4.2 Research Design Target Participant Sample: As we chose Solid States as our case-study topic, it became very obvious for us to define our target sample group as chemistry students of class 11th and12th also with the students who drop one year after 12th class for college entrance exams. Variables: Our single independent variable will be the manner in which content is delivered to the students. Basically, we will try to compare these different manners of content delivery and study the effects of them on dependent variables. We are planning to use Single way Multivariate ANOVA (Analysis of Variance) test to analysis purpose. There are four levels of this independent variable: 1) Traditional face-to-face classroom setting in which teacher use either printed NCERT books and physical 3d models (mostly balls) to teach the students Solid State concepts. 2) Video: Videos can also be used to explain the concepts. There can be different types of videos also other than basic camera recorded video: Interactive or Animation videos 3) Mouse controlled 3d navigation web apps 4) Augmented Reality (AR) based solution: 3d rendered objects are projected onto markers which are tracked by the device camera. In contrast with mouse controlled apps, these are easier to learn and also give sensorimotor feedback while using it. Navigation from one view from another is easy and quicker. There is more directness in interaction with 3d object in case of AR based solution. We would study the effects of above different levels on following dependent variables : 1) Course Performance 2) User Perceived Satisfaction 3) User Engagement
  20. 20. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 21 4.3 Design Guidelines In [Wu and Shah, 2004], authors have suggested five principles for designing chemistry visualization tools that help students understand concepts and develop representational skills through supporting their visuospatial thinking. These principles are as following: (i) Providing Multiple Representations and Descriptions: As students faces difficulty in representing chemical concepts at the microscopic and symbolic levels, comprehending representations conceptually, it becomes important to provide them the representations in multiple along with descriptions. (ii) Making Linked Referential Connections Visible: Second principle is to make linked referential connections among representations visible so that students could construct appropriate conceptual connections among multiple representations. One way to help students visualize the connections is to allow a representation to be changed by manipulating its connected representation or description. (iii) Presenting the Dynamic and Interactive Nature of Chemistry: Students generally face difficulty in visualizing the movement of particles and develop a dynamic model of chemical processes. The dynamic mental models developed via viewing animation or series of static diagrams could help students learn advanced chemical concepts and enhance their visuospatial thinking. (iv) Promoting the transformation between 2d and 3d: Fourth design principle is to provide features that facilitate the identification of depth cues and the transformation between 2D and 3D. (v) Reducing Cognitive Load by Making Information Explicit and Integrated: Reducing cognitive load is an important factor for making visualization tool helpful for student with low visuospatial abilities. This can achieved by providing visual and verbal information contiguously rather than separately.
  21. 21. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 22 4.4 User Requirement Analysis We conducted user research with the aim to identify the problem points and needs of teachers and students. Also, we intended to select few concepts from Solid States chapter for development purpose on the basis of insights from user research. With these objectives in mind, we had semi-structured interviews with five higher secondary class chemistry teachers. Teacher School/ Coaching Current Organization Interview Method City A School Kendriya Vidyalaya Physically Guwahati B School Mount Carmel Virtually Delhi C Coaching Concept Education Physically Guwahati D Coaching Oriental Tutorials Physically Guwahati E Coaching FIITJEE Virtually Delhi
  22. 22. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 23 4.4.1 Interview Questionnaire We had six subjective questions in our questionnaire as follows: 1) Do you find any relative difference in teaching concepts of Solid States in comparison to other chapters? 2) As a part of your teaching curriculum, what is the standard division of the chapter - could you please divide the chapter into subtopics and modules based on your teaching techniques For example, if you cover the chapter in a span of 3 classes, which topics are broadly covered in which of the classes 3) Within these modules, are there any specific topics which are relatively difficult to explain / teach / make students understand? 4) From a student's perspective, what are the topics within the chapter in which they face maximum difficulties / find hard to grasp? 5) Is NCERT content sufficient to explain all concepts of Solid States in a concise manner? Is there any other reference material that is recommended to students? 6) Do you feel need of or use any additional visualization tools to explain the Solid States concepts to students more constructively? If yes, what could be they?
  23. 23. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 24 Figure 12: Interview at Oriental Tutorials, Guwahati 4.4.3 Insights from Interviews 1. Difference between Solid States and other chapters: Responses to this question are quite consistent for all five teachers. They describe Solid States chapter as more demanding in terms of 3 dimensional visualization and imagination for students. Correlation among views of different teachers can be easily seen in their statements. One teachers said, “As solid states involves 3d concepts, it requires more visualization and imagination skills of the students”. According to another teacher: “It gives help to understand 3-D structures of metals and Ionic Compounds. Visualization in 3-D is required.” These feedback gives support to our assumption that there is need of 3d visualization aiding for students in Solid States and nurture our motivation to design a Augmented Reality based tool for the same. 2. Division of chapters into different modules and sub-topics: As some teachers are more focused towards teaching school syllabus whereas other are focused towards teaching entrance exam syllabus. Therefore, there are slight differences across teachers in the content and the modules in which the content is divided. Even though, there is similarity in terms in terms of teaching core concepts of the chapter: different layer wise 3dimensional arrangement of atoms, unit cells of Face Centered Cubic (FCC) and Hexagonal Closed Packing (HCP) and tetragonal and octahedral voids. We also asked from some of the teacher’s most important topic in the chapter. These insights helped us to choose spatial arrangement of atoms in unit cells and voids formed inside them as content for AR based pedagogical tool to start with.
  24. 24. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 25 Figure 13: Interview at Kendriya Vidyalaya, IIT Guwahati 3. Relatively difficult topics to teach and learn Teachers find it difficult make student visualize and understand the spatial arrangement of particles in 3d space. One teachers informed, “For students it is difficult to understand 3d crystalline structure and where and how different voids are present inside the structures.” From different structures couple of teachers found Hexagonal cubic packing relatively difficult to visualize and so to teach. A teacher said, “In hexagonal packing, visualization is bit difficult and then voids in hexagonal packing.” Solid States chapter contains other concepts also e.g. Voids, Cation-Anion Ratio, Coordination number. There are numerical problems in these concepts. These concepts are associated with and extension of basic concepts of 3d structure arrangement and unit cells. According to one teacher, “Once 3d arrangement of atoms is clearly understood by student, everything else is easier.” This information motivated us to start with spatial arrangement of atoms in unit cells and voids as instructional content. 4. NCERT is insufficient Most of the teachers admire NCERT text books because of the content and instruction design. It somewhat helps students understand the crystalline structure with the help of colorful 2d figures. But they do not find it sufficient in terms of depth of content and its effectiveness in provide clear 3d visualization of structures and lattices. One teacher stated, “NCERT books are good and there are some diagrams and explanations for 3d concepts but not sufficient.” They generally refer foreign author books or other guide books. 5. Use of additional tools Teachers take help of ball - stick models and animations to show how molecules are arranged in a unit cell and voids are created. One teacher provided us with the details of the tools he has used. He
  25. 25. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 26 informed, “I tried the following ball stick models: Deluxe Version Solid State Model Kit (http://ice.chem.wis c.edu/Catalog/SciKi ts.html#Anchor-Solid- 31140). Currently I am using bits of J3D animation from http://www.chm.davi dson.edu/vce/ which are extremely effective and students just enjoy them.” There were opposite views also. 3d physical models could be difficult to make, store or carry. According to one teacher, “It is time consuming to make slides or use 3d models. There is non availability of 3d models in market.” Also, these models are just static 3d representation of one state of lattices. Animations are again dynamic 2d representation of crystalline structure. Another teacher shared his views, “Unfortunately the videos and models are not very useful and user friendly so they also do not provide much help for teachers. If we can have the visualization of the 3-D structure that how a structure is formed step wise it will help. It should be handy and simple to use.” It was interesting to find that most of the teachers use example of room to teach arrangement of atom in cubic unit cell and sharing among different unit cells.
  26. 26. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 27 Figure 14, 15: D Fusion Studio Chapter 5: Development 5.1 D’Fusion Initially, we did some explorations with D’Fusion studio, a cross platform SDK for building AR applications by Total Immersion. It is more GUI based and one can develop basic AR applications (augmentation of single 3d rendered supplement onto real world by tracking single marker) without much programming. Scenario intelligence programming is done using Lua script. 3D rendered objects can be directly imported from Autodesk 3ds Max and Maya using exporters provided in its developer package. We were successful in augmenting 3d molecular structure over black and white marker. We also tried adding interactivity to it by changing the rendered supplement when two markers are brought nearby. But during the course of our exploration with D’Fusion studio, we found following issues in it: 1) Marker-Tracking is very unstable, a lot of flickering was occurring while tracking. 2) It shows its trademark logo all the time over display screen. 3) Interactive elements like on screen buttons and animations were difficult to add. 4) Weak developer community and support. 5) One have to do a lot of steps just for basic augmentation Due to these issues, we decided not to proceed with D’Fusion and switched to Vuforia.
  27. 27. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 28 Figure 16: Vuforia by Qualcomm Figure 17: Unity software Figure 18: SketchUp software 5.2 Vuforia, Unity, SketchUp Vuforia by Qualcomm is an Augmented Reality Software Development Kit (SDK) for mobile devices that enables the creation of Augmented Reality applications. It uses Computer Vision technology to recognize and track planar images (Image Targets) and simple 3D objects, such as boxes, in real-time. This image registration capability enables developers to position and orient virtual objects, such as 3D models and other media, in relation to real world images when these are viewed through the camera of a mobile device. The virtual object then tracks the position and orientation of the image in real-time so that the viewer’s perspective on the object corresponds with their perspective on the Image Target, so that it appears that the virtual object is a part of the real world scene. Apart from providing Image tracking capabilities, Vuforia also gives developers the flexibility to add interactions through buttons, gestures, animation, sound etc. in the mobile application. Tracking is very stable in Vuforia in comparison with D’fusion. Programming in Vuforia is done on C sharp and Java script with unity. SketchUp, marketed officially as Trimble SketchUp, is a 3D modeling program for applications such as architectural, civil and mechanical engineering, film, and video game design. It provides an intuitive graphical user interface to design 3D cad models similar to softwares such as 3DS Max, Rhino etc. Unity is a cross-platform game engine with a built-in IDE developed by Unity Technologies. It is used to develop video games for web plugins, desktop platforms, consoles and mobile devices. Unity is of extreme importance to this project because it provides a base platform to use 3D models generated in Sketchup with the Vuforia plugin. Additional functionalities and interactions such as GUI buttons, audio support and virtual buttons can be built on top of this using Unity.
  28. 28. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 29 Figure 19: Virtual Buttons (in blue) and GUI buttons (in black) 5.3 Virtual Buttons and GUI 5.3.1 Virtual Buttons Virtual buttons are developer-defined rectangular regions on image targets that trigger an event when touched or occluded in the camera view. For example, in the sample picture, pointing the hand or touching the rectangular region triggers an action associated with the button. Such buttons provide an intuitive means of interaction since the users are directly using the content (on paper / surface) to navigate / as a button rather than on screen buttons 5.3.2 GUI The graphical user interface of Augmented Reality Apps are primarily simple because a major chunk of screen space is dedicated to the camera for easy viewing. Any additional content that needs to be shown to the user is subsequently placed on layers above the camera layer. In this project, we have used two GUI buttons to allow users to navigate / toggle between different views of the same 3D model. The models are placed in a chronological order - i.e, the next view of the model is obtained from the previous view.
  29. 29. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 30 Figure 20: Task Flow Diagram, Module 1 5.4 Application We divided our teaching into two modules, based on the content finalized through feedback from our qualitative research. These modules are: 1. Understanding 3D Closed Packing Structure 1a. Hexagonal Close Packing 1b. Cubic Close Packing 2. Understanding Voids 2a. Tetragonal voids 2b. Octahedral voids 5.5 App Flow The flow of the app can be understood through the following steps: 5.5.1 Module 1 1. User is reading the NCERT book and comes across the concept of 3 Dimensional closed packing. 2. User turns on the application on his mobile / tablet 3. The home screen of the application is essentially live feed from the camera of the device. The user points the device to the page of the NCERT book. 4. The 3D model is augmented on the device with audio feedback. Virtual buttons to toggle between hexagonal close packing and cubic close packing are also augmented on the device. This 3D model consists of two layers of atoms in which placement of second layer is shown through animation. The first layer is white in color while the second is in green. Different colors are used to for different orientations of layer and easy understanding. 5. The user points / touches the desired concept to be explored on the NCERT book. 6. Subsequently, the animation and placement of third layer is shown
  30. 30. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 31 Figure 21: App Screenshots, Module 1 6. a Hexagonal Close Packing In case of hexagonal close packing, the third layer is positioned exactly the same way as the first layer, forming ABAB structure. The placement of third layer (white in color, same as first layer) is shown through animation upon selection of hexagonal close packing through the virtual button on the book. Also, once the user selects hexagonal close packing, two GUI buttons appear on screen (image here) namely ‘Next’ & ‘Back’. These buttons can be used to navigate back and forth to subsequent views of this packing. In the next view (image here), additional atoms from each layer are removed leaving out just one unit cell, to be able to visualize the hexagon formed through such a packing. In the subsequent view, a a translucent hexagon is augmented over the atoms to show how the unit cell looks. Each of these steps is accompanied with audio feedback explaining the concept and providing concepts. Finally, for effective learning of these concepts, the user is prompted with a question related to the concepts shown in the previous slides in the form of a multiple choice question. In case a user answers correctly, the user is prompted again with a question about reasoning of the correct answer / why other options were incorrect. Only upon correctly answering both these questions is the user shown an explanation about the actual answer of the question. Such a twofold system of testing ensures that the student approaches a problem from different perspectives and identifies different use cases (For example, visualization of layering of atoms in a different fashion / orientation). It also helps complete the learning cycle of the concept being communicated through the application.
  31. 31. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 32 Figure 22: App Screenshots, Module 1 Figure 23: AppTest Screenshots, Module 1 6. b Cubic Close Packing In case of cubic close packing, the third layer is not aligned either with the first layer or the second layer. Thus, the third layer has its own color (blue) the atoms of which are placed such that they fit into the octahedral voids formed by the previous two layers. When the user selects cubic close packing through the virtual button, placement of this layer is shown through animation over the first two layers. Also accompanying the third layer is the fourth layer in white, which is aligned exactly with the first layer, thereby forming ABCABC layering of Cubic close packing. Similar to hexagonal close packing, upon selected of CCP through the virtual button, two GUI buttons appear on screen (image here) namely ‘Next’ & ‘Back’. These buttons can be used to navigate back and forth to subsequent views of this packing. In the next view (image here), additional atoms from each layer are removed leaving out just one unit cell, to be able to visualize the cube formed through such a packing. In the subsequent view, a a translucent cube is augmented over the atoms to show how the unit cell looks. Each of these steps is accompanied with audio feedback explaining the concept and providing concepts. This particular visualization of a cube is of importance to us since it involves rotation of the atoms at an angle which is difficult to visualize. The color coding used layers wise accompanied with freedom to spatially move in 3D helps students correlate this form of ccp to the 1st state (ABCABC) The user can navigate back to any of the previous views through on screen buttons. The user can also navigate to other concept (Cubic Close Packing) through virtual button. Also, these models of CCP are accompanied by a test question, followed by a question on the justification of incorrect options.(Similar to the model followed in hexagonal close packing).
  32. 32. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 33 Figure 24: Task Flow Diagram, Module 2 5.5.2 Module 2 Understanding Voids Voids are the empty space created between atoms when they arranged very nearby. For students, understanding different kind of voids, how they are formed, their 3d positions in single unit cells and how they are shared between multiple unit cells are very important. In ionic crystalline solid structures cations are present on voids. Therefore, to calculate cation anion ratio in a molecule, it is important to know above mentioned details about voids. Therefore, in our second module we chose voids in Face Centred Cubic (FCC) as our content material. In a Face Cantered Cubic unit cell, there are atoms at each corner of the cube as well as on the centre of each face. There are two type of voids in FCC: (i) Tetragonal Voids (ii) Octahedral Voids. These voids in FCC unit cell are described on page 17 of 12th class Chemistry NCERT book. There are two diagrams on the page: upper one for tetragonal voids and lower one for octahedral voids. When student starts the Clearn (AR application) and bring the camera in front of the page 3d model of FCC is augmented on the screen. Also, there are two virtual buttons on the page, one on each diagram and so for void type. Student can choose to learn any of the void concept by point towards desired virtual button. Tetragonal voids A tetragonal void is formed by placing fourth atom over the depression among three closely arranged face centred atoms. Initially, all atoms of FCC unit cells are colored grey. When tetragonal void’s virtual button is pressed, the four relevant atoms are colored orange to distinguish them from other molecules. These four atoms are joined and four triangular green translucent faces are shown to form the tetrahedron. Other than these changes in 3d model, ‘Back’ and ‘Next’ are also shown on the screen. Student can toggle between different steps/models using these buttons. By pressing next button small green sphere is shown at exact center of the tetrahedron. This sphere abstractly represent the position of tetrahedral void. So, tetrahedral voids are present on the one-fourth of the body diagonal of
  33. 33. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 34 Figure 25: App Screenshots, Module 2 FCC unit cell. In Sodium Oxide, Sodium atoms in green are placed at these tetrahedral voids. On pressing next button, all 8 tetragonal voids are shown as green spheres and all other spheres are turned into orange. Instructional audio related for each mode is also being played. Octahedral voids Whenever three closely packed atoms are placed directly over three oppositely oriented atoms, an octahedral void (OV) is formed within them. There are two types of such voids in fcc unit cell. The first formed at a body center is shown here. When octahedral void’s virtual button is pressed, octahedral void at body center of FCC unit cell is shown with three spheres of same layer as blue and other three as orange. On pressing next button, second type of octahedral void, edge centered void is shown. This time four unit cells are shown and one edge centered OV is shared among these four unit cells. After pressing next button, small red sphere is appeared on the exact center of the octahedron formed by 6 face centered atoms around center of unit cell. This sphere abstractly represent the position of octahedral void. In Sodium Chloride, Sodium atoms in green are placed at these octahedral voids. On pressing next button, all 13 positions of octahedral voids are shown which due to sharing of edge centered atoms are effectively four. Instructional audio related for each mode is also being played.
  34. 34. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 35 Figure 26: 3D Models, Module 1 5.6 Audio components To assist learning and provide instruction, audio feedback was added into the application to guide users through the flow of the application as well as help in instruction. A mute button to turn of these instructions has also been provided on the GUI. The following is the audio feedback given by the application at respective stages: Module 1 : Understanding 3D Closed Packing Structure Stage 1 (Layer 1 + Animation of Layer 2 on top of it) “3Dimensional close packed structure can be generated by placing layers one over the other. Let us take a two dimensional hexagonal close packed layer ‘A’ colored in white and place a similar layer colored in green above it such that the spheres of the second layer are placed in the depressions of the first layer. Let us call the second layer B. For placement of the third layer, point your finger at either the diagram of hcp or ccp on your NCERT book (Figure 1.18 b)” Stage 2a User selects hcp virtually “In Hexagonal close packing, tetrahedral voids of the second layer in green are covered by the spheres of the third layer in white, which is aligned exactly with the first layer. Thus, the pattern of spheres is repeated in alternate layers and is often written as ABAB. Toggle between different visual modes by on screen buttons.” Stage 2b User toggles to next mode (hcp) “One unit cell of such hexagonal close packing can now be seen after removal of atoms of other cells from each layer.”
  35. 35. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 36 Figure 27: 3D Models, Module 1 and 2 Stage 2c User toggles to final mode (hcp) “The faces of this hexagonal unit cell can now be seen. This sort of arrangement of atoms is found in many metals like magnesium and zinc.” Stage 3b User selects ccp virtually “In Cubic close packing, octahedral voids of the second layer in green are covered by the spheres of the third layer in blue. When placed in this manner, the spheres of the third layer are not aligned with those of either the first or the second layer. Only when fourth layer in white is placed, its spheres are aligned with those of the first layer from which the pattern ABCABC emerges. Toggle between different visual modes by on screen buttons.” Stage 3c User toggles to next mode (ccp) “One unit cell of such cubic close packing can now be seen after removal of atoms of other cells from each layer. “ Stage 3d User toggles to final mode (ccp) “The faces of this cubic unit cell, known as face centred cubic can now be seen. Note how the original layers are oriented within a cubic cell. Metals such as copper and silver crystallise in this structure.” Module 2: Understanding Voids Stage 1: Cubic model “In a Face Centered Cubic arrangement, there are atoms at each corner of the cube as well as on the centre of each face. Point your finger at Figure 1 or Figure 2 to know more about tetrahedral or octahedral voids respectively.” Stage 2a: Tetragonal void is selected “Tetragonal void is selected. A regular tetrahedron is formed connecting three face centred atoms and one atom at the corner of the unit cell
  36. 36. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 37 Figure 28: 3D Models, Module 2 (Orange in color). This tetrahedron is actually the tetragonal void within the four atoms. Toggle between different visual modes by on screen buttons.” Stage 2b: Next mode of tetragonal void “Within this tetragonal void formed inside the tetrahedron, an atom can be placed. For example, in Sodium Oxide, Sodium atoms in green are placed at these tetrahedral voids.” Stage 2c: Final mode of tetragonal void “A total of 8 such tetragonal voids are thus formed in each fcc unit cell, as shown.” Stage 3a: Octahedral void is selected “Whenever three closely packed atoms are placed directly over three oppositely oriented atoms, an octahedral void is formed within them. There are two types of such voids in fcc unit cell. The first formed at a body centre is shown here.” Stage 3b: Next mode of Octahedral void “Octahedral voids are formed on the center of the edges as well. It can be seen that one edge centered octahedral void is shared amongst four unit cells.” Stage 3c: Next mode of Octahedral void “Within this octahedral void formed inside the octahedron, an atom can be placed. For example, in Sodium Chloride, Sodium atoms in red are placed at octahedral voids.” Stage 3d: Final mode of Octahedral void “Effectively there are 4 such octahedral voids formed in each fcc unit cell.”
  37. 37. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 38 Figure 29: Students using the prototype Chapter 6: Initial Feedback The prototype developed was tested for qualitative feedback at Kendriya Vidyalaya, IIT Guwahati amongst class XII children. Aim of this study was to get the initial feedback of concept and prototype from its primary users i.e. students, identify the major shortcomings in them and then look for the scope for improvement. Some key insights from this study are:  Wow factor and non familiarity with technology major driving force behind initial feedback.  Some students pointed that they would have liked to see rotation and movement through touch gestures on phone as well.  One student wanted content to be broken down to even smaller steps (atom joining atom instead of layer joining layer)
  38. 38. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 39 Figure 30: Classification of A.R publications by evaluation method / approach Chapter 7: Literature review (Phase II) 7.1 Evaluation Techniques Although Augmented Reality (AR) has been in studied for over forty years it has only been recently that researchers have begun to formally evaluate AR applications. Most of the published AR research has been on enabling technologies (tracking or displays, etc.), or on experimental prototype applications, but there has been little user evaluation of AR interfaces [Dunser, et.al, 2007]. Existing literature [Dunser, et.al, 2008] indicates that AR user evaluation papers can be classified into five types: (1) Objective measurements (2) Subjective measurements (3) Qualitative analysis (4) Usability evaluation techniques (5) Informal evaluations Objective measurements include task completion times and accuracy / error rates; other examples are scores, position, movement, number of actions, etc. In general these studies employ a statistical analysis of the measured variables, however, some only include a descriptive analysis of the results. Subjective measurements are those in which users are studied using questionnaires, subjective user ratings, or judgments. With respect to analysis some of these studies also employ statistical analysis of the results, others only include a descriptive analysis. Qualitative analysis category comprises studies with formal user observations, formal interviews, or classification or coding of user behavior. Usability evaluation techniques are those that are often used in interface usability evaluations such as heuristic evaluation, expert based evaluation, task analysis, think aloud method, or Wizard of OZ method. Lastly, informal user evaluations are those that include informal user observations or informal collection of user feedback. It has been observed that the ratio of formal user evaluations compared to
  39. 39. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 40 informal evaluations has increased over the years. Between 1995 and 2001 there is an average of 57% formal evaluations, whereas between 2002 and 2007 this percentage is 76%. Thus there seems to be a growing understanding for the need to formalize the evaluation process and conduct properly designed user studies. 7.2 Spatial Ability Spatial ability can be described as the ability to picture three-dimensional (3D) shapes mentally. [Martin et. al, 2010]. Educational research of Potter and Vander Merwe [Potter, 2003] concluded that spatial ability influences academic performance in engineering. But, every student in the classroom doesn't have a good spatial ability. Previous studies have shown that students with lower visuospatial abilities are unable to (i) perform well in solving spatial and non- spatial chemistry problems [Bodner & McMillen, 1986; Carter, LaRussa, & Bodner, 1987], (ii) identify the depth cues of 2D models [Seddon, Eniaiyeju & Chia, 1985],(iii) form 3D mental images by visualizing 2D structures [Tuckey, Selvaratnam & Bradley, 1991] and (iv) comprehend symbolic and molecular representations conceptually [Ben-Zvi, Eylon, & Silberstein, 1988]. There are multiple studies which divides spatial ability in sub-domains [Guttman et. al, 1990; Lohman, 1979]. These two factors have consistent across these studies: Spatial Relations which is speeded mental rotation and Spatial- Visualization which includes all complex, multi-step spatial tasks [Lohman, 1979]. Tasks involving three-dimensional mental rotation are somewhat intermediate and have been grouped into each of these two factors. Lohman, 1979] Tasks requiring participants to imagine different perspectives either form a third factor or are grouped into Spatial Relations. Mental Rotation Test (Vandenberg, 1978) is a popular test to assess spatial relation skills whereas spatial visualization can be assessed by
  40. 40. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 41 Figure 31: Technology Acceptance Model Purdue Spatial Visualization Test (Guay, 1977). 7.3 TAM Technology Acceptance Model (TAM) one of the most widely accepted model which explains the relations between user attitudes, satisfaction and behavioral intention to use the information systems. [Davis, 1989] first introduced the TAM as a theoretical extension of the theory of reasoned action (TRA) [Fishbein and Ajzen, 1975]. This model predicts user acceptance based on the influence of two factors: perceived usefulness and perceived ease of use. Perceived usefulness is defined as ‘‘the degree to which a person believes that using a particular system would enhance his/her job performance’’, and perceived ease of use is defined as ‘‘the degree to which a person believes that using a particular system would be free of physical and mental effort’’ [Davis, 1989]. TAM posits that user perceptions of usefulness and ease of use determine attitudes toward using the system which further determines the behavioral intentions, in turn leading to actual system usage. TAM has been extended by addition of other constructs called external variables which perceived usefulness or perceived ease of use such as self-efficacy [(Compeau and Higgins, 1995], subjective norm [(Taylor and Todd, 1995] or playfulness [Moon and Kim, 2001]. Davis's original proposition of TAM has more 1000 citations. Several attempts have been made in the past by researchers to consolidate the results from these studies in terms of meta- analysis [Yousafzai et. al., 2007; King and He, 2006]. There are abundance of studies which confirms TAM to be a good theoretical tool to understand users’ acceptance of e-learning [Lee, Cheung and Chen, 2005; Park, 2009]. [ŠUmak et. al., 2011] have presented meta-analysis of TAM studies in context e-learning technologies. [Park, 2009] suggests e- learning self-efficacy and subjective norm as important factors to determine
  41. 41. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 42 Figure 32: Sample PSVT Question attitude and behavioral attention towards e-learning. TAM has also been used check the acceptance of mobile augmented reality application with historical photographs and information about a historical street [Haugstvedt et. al., 2012]. The results show that both perceived usefulness and perceived enjoyment has a direct impact on the intention to use such mobile augmented reality applications. 7.4 PSVT Purdue Spatial Visualization of Rotations Test (PSVT:R) is a common test to measure spatial visualization ability of chemistry students [Bodner, Guay, 1997; Carter, LaRussa, & Bodner, 1987]. Actual PSVT [Guay, 1977] consisted of three sections: Developments, Rotations and Views. Developments consisted 12 questions designed to see how well subjects can visualize the folding of developments into three-dimensional objects. Rotations consisted 12 questions designed to see how well subjects can visualize rotations of three- dimensional objects. Rotations consisted 12 questions designed to see how well subjects can visualize what three-dimensional objects look like from various viewing positions. There were also 30-items test booklets: one each for Developments, Rotations and Views. Out of these 30 questions on rotations, [Bodner and Guy, 1997] removed question 6, 8, 11, 14, 20 ,21, 22, 24, 26 and 30 to reduce it to item-version. One item from 20-item PSVT test is shown in Figure. In this test, participants view two rotated versions of one 3D figure, infer the type of transformation between them, and make the same transformation with a new 3D figure.
  42. 42. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 43 7.5 VARK A learning style or preference is the complex manner in which, and conditions under which, learners most efficiently and most effectively perceive, process, store, and recall what they are attempting to learn [James & Gardner, 1995]. One characterization of learning styles is to define the learners’ preferred mode of learning in terms of the sensory modality by which they prefer to take in new information. VAK is an acronym that stands for three major sensory modes of learning: visual, aural, and kinesthetic, depending on the neural system with which a learner prefers to receive information. Thus VAK is a perceptual, instructional preference model that categorizes learning by sensory preferences. Recently, Fleming [Fleming, 1995] expanded VAK to VARK to include reading/writing as an additional type of mixed sensory learning modality. Although learners can use all of these sensory modes of learning, one mode is often dominant and preferred. For example, visual learners learn through seeing drawings, pictures, and other image-rich teaching tools. Auditory learners learn by listening to lectures, exploring material through discussions, and talking through ideas. Reading/writing learners learn through interaction with textual materials, whereas kinesthetic learners learn through touching and experiences that emphasize doing, physical involvement, and manipulation of objects. Students have preferences for the ways in which they receive information. The visual, auditory, reading/writing, kinesthetic (VARK) questionnaire identifies student’s preferences for particular modes of information presentation. The following are internet links to the VARK homepage (http://www.vark- learn.com/english/index.asp) and questionnaire (http://www.vark- learn.com/english/page.asp?pquestionna ire). We administered the VARK questionnaire to our participants as a part of our pre questionnaire to be able to draw inferences with their learning styles and performance in spatial visualization tests to be conducted as a part of our main experiment.
  43. 43. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 44 Figure 33: Octahedral void as seen in new prototype Chapter 8- Improvizations in AR prototype 8.1 Introduction Based on the initial qualitative feedback received after showcasing our application to high school students, teachers & professors from our institute (as discussed in chapter 6), we decided to incorporate several changes in our application. These improvisations were completed before proceeding with our experiment design which involved testing through a comparative analysis with the web counterpart of the applications. Also, since the comparative analysis involved an experiment design that needed to completed within a fixed time, we narrowed down our content even further to voids - tetrahedral + octahedral.
  44. 44. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 45 Figure 34: Removal of virtual buttons and changes in GUI 8.2 Changes The improvisations in the application include: 1) Removal of virtual buttons Virtual buttons were used in our system since they provided context specific use - i.e, by pointing at a particular content on the physical book, related content used to be augmented on or application. Our initial testing suggested that virtual buttons were a hindrance for the users since they had to switch their focus between the screen and the textbook regularly. While operating the application the focus of the users is on the screen of the tablet / mobile where the content is displayed. However, when the user has to choose a virtual button, he needs to shift focus back on the book and regularly switch between the tablet and the book to be able to select the virtual button. Also, since the virtual buttons were placed as per the diagrams (content) in the book, more than often, these buttons were in close proximity to each other and of smaller size. This resulted in tracking issues since instead of the finger sometimes the hand / arm used to false trigger an option. Also, when the hand was brought on top of the book to choose a virtual button, the main image tracker was also obstructed. Keeping these points in mind, we decided to replace virtual buttons by on screen GUI buttons so that user’s concentration is not diverted at any point of time and there is no limitation because of tracking errors and issues. 2) Changes in GUI Since the virtual buttons were removed, new buttons had to be added into the GUI to provide the same functionality. The option to choose between tetrahedral void and octahedral void was provided to users in the 1st screen where an FCP model was shown. Once an option was selected, the users now had three options - to proceed to the next model within the selected category (Tetrahedral / octahedral) or to switch to the other category. These three buttons were grouped together in conjunction with the law of proximity (reference) since all of them had similar functionality of navigating between content .Apart from
  45. 45. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 46 Figure 35: Addition of interactivity by touch this group, there was another button placed at a distance for audio control. 3) Changes in Audio controls Initially, the audio button had the functionality of mute - i.e, the audio used to play automatically and the users had the option to mute it. The audio still used to keep playing in the background but was not audible. We observed that the majority of the users preferred to mute the audio in the beginning since they were concentrating initially on the augmented model and the interactions surrounding it. After exploring the model for a while, when they unmuted the audio, the file had already played for a significant amount of time and it difficult for users to pick up from mid-way. In the new interface, the audio did not play automatically in the start. Instead, the users had the option to tap on Play audio to begin listening to the audio content as and when they wanted to as per their convenience. Also, instead of providing of pause, the play button transformed into a stop audio button once the play was pressed. The reason for choosing stop over pause is two fold . The first reason is as discussed before - the difficulty faced by users in grasping content mid-way. The second reason is that if users paused an audio and moved to some other model for exploration, upon returning to the original model it was all the more difficult to be able to understand the audio by resuming mid- way. 4) Addition of interactivity by touch The third major change in the interface was the added interactivity of rotation of models through swipe on the screen. A majority of the users being accustomed to touch screen devices expected to be able to rotate the model through such an on screen interaction. Also, this removed the constraint of not being able to view a 3D model from below, as the model could be rotated. The users now could rotate the model as per their convenience by swiping in the direction of rotation. The swipe rotation also included inertia so that the rotation looked more natural - i.e, upon swiping in a particular direction, the model rotated for a particular angle and then came to a smooth stop based on the speed of the
  46. 46. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 47 Figure 36: Zoomed in view of prototype swipe. A single tap on a rotating model also brought it to a halt. 8.3 Final GUI walkthrough: The final application thus has a total of eight 3D models with their associated audio files. The application begins when the camera tracks the NCERT page on voids. An FCP model is then augmented on the surface. A total of 3 GUI buttons appear on the top - two grouped together (option to choose tetrahedral or octahedral) and the third being that of the audio. Upon selecting either tetra / octa, the GUI shows the respective model augmented on the NCERT. The buttons are now changed - now grouped in three. These buttons are that of previous model, next model or the option to switch between tetrahedral or octahedral. The play audio button is common throughout and can be used to play audio content related to the model being augmented. The users can pan through by moving around the tablet to view the augmented model from all sides and angles. Bringing the tablet closer to the NCERT booklet serves as a zoom in and allows user to explore the models from a closer angle. Similarly, taking the tablet away serves as zoom out. Also, apart from moving the tablet device for zoom / pan, the NCERT book or the image tracker itself can be moved, brought closer or rotated to serve pan / zoom features. Since the model that is augmented is fixed to the tracker, moving the tracker also moves the model. Lastly, the users can use on screen swipe gesture to rotate the model in any direction they wish to.
  47. 47. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 48 Figure 37: PSVT A.R Prototype Additional application: Apart from these changes in our application, we also developed another standalone application to be used for conducting PSVT as a part of our comparative analysis experiment. For this application, we used a standard pebble image default tracker provided by Vuforia. The application consisted of each of the twenty 3D models given as a part of the PSVT. These are the models which users have to finally rotate as per the sample example given in the question. The interface consisted of two GUI buttons - previous question and next question, which allowed them to navigate between these 20 models. The feedback about which model / question they were currently on was also provided on screen. The users thus had the support of this system to help them answer the PSVT questionnaire - they could see in 3D the model asked via the questionnaire and also pan & rotate the model via finger swipe to be able to help them visualize the rotated view as asked in the question.
  48. 48. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 49 Chapter9 - Web Interface 9.1 Introduction Along with Augmented Reality (AR) based e-learning tool: Clearn, we designed and prototyped a conventional web based tool as well. Reason behind having this interface is to compare Clearn with web based tool and find the issues in it. This web interface is completely graphical user interface based, contains multiple interactive 3d object viewers and controlled by computer mouse.
  49. 49. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 50 Figure 38: Web Interface (Voids) 9.2 Design Web interface basically contains series of multiple HTML pages linked together. Each page contains a 3d object viewer in which 3d digital model can be rotated in any direction using mouse dragging. There is also option of zooming in and out the model (through mouse scroll/wheel) along with full screen view. We emphasized on keeping the two interfaces (AR and Web) as similar as possible in terms of interactions to avoid any confounding variable i.e. effect on results due to some extra feature. We achieved so by taking following design decisions for web interface: (i) Visual design of web interface is kept minimal having white background and no extra visual element (ii) Number of buttons, labels on them and their functions are same as AR interface (iii) Maintaining the consistency with AR interface in terms of 3d models and audio instructions (iv) Task flow of the interface is also similar to that of Clearn interface Similar to AR case, we designed and developed two web apps: one for Solid State concepts of voids and another for Purdue Spatial Visualization Test of Rotations (PSVT: R) assistance. In solid state web-app, one can explore different face centered cubic unit cells having voids and navigate through these models using GUI buttons. Second app was intended to aid participants while attempting PSVT: R. Interface is shown in the figures.
  50. 50. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 51 Figure 39: Web Interface (PSVT) 9.3 Development Web Interface was developed by programming in HTML and CSS. Basic layout of the webpage was developed using bootstrap framework (http://getbootstrap.com/). HTML 5 audio player was used to add audio instruction feature. 3d object viewers were embedded in webpages using Sketchfab (https://sketchfab.com/) and p3d (http://p3d.in/) for solid state and PSVT respectively. These are web- services which allows you to upload your 3d models online and then embed them on your web-pages. There is restriction of rotation of models along vertical direction after a particular angle in Sketchfab. Therefore, we chose to use p3d for PSVT web tool because flexibility in rotation along all directions is very important while solving PSVT. We uploaded the Sketchup models same as AR apps.
  51. 51. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 52 Chapter 10 - Research Methodology 10.1 Aims & Rationale Identifying the areas where Augmented Reality based tool lacks in comparison with conventional tools of e-learning, is the main aim of this study. For this, we decided to compare a web based e learning tool with an augmented reality e learning tool in a controlled experimental setting. We compared the learnability of students through questions that involved spatial visualization and deep understanding of the content. In this, we expected students using A.R based e learning tool to perform better as compared to those using a web based e learning tool. We believed that the familiarity and comfort in usage of web based systems would be outweighed by the novelty and better 3D viewing in A.R which would result in better understanding and visualization of 3D content. We also compared the effect of such systems on spatial rotation skills. The goal was to discover whether interactive 3D applications both on web and A.r would support similar level of spatial skills to traditional mental rotation scenarios. Apart from these, the goal also was to compare web with A.R to discover if there was any difference in their support to spatial skills. Because of the familiarity of users with mouse based interactions, we expected web based e learning tool to perform better in terms of time taken to complete the task. However, given same freedom to rotate and view the models in both the systems, we expected no significant difference in the accuracy with which the questions are answered. Lastly, we compared both the A.R and web systems on six parameters using the technology acceptance model - perceived usefulness, perceived ease of use, attitude, behavioral intention, self- efficacy and perceived enjoyment. We believed both the systems to perform equally well on all parameters, with A.R performing slightly better in terms of enjoyment. The novelty and the halo effect associated with augmented reality was expected to increase its enjoyment scores.
  52. 52. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 53 10.12 Experiment design We conducted a between-subjects study between group using AR tool and group using web based tool due to two reasons: first, to avoid issue of creating 2 content quiz of same difficulty level and second, reduce the experiment time for each participant. Therefore, our independent variable was e-learning tool with two levels: Augmented Reality based tool and web based tool. We did not incorporate a gender variable since it is common in spatial skills studies and was not the focus of the study. Our dependent variables were accuracy of content related questions (scale, in % of correct answers), response time for PSVT (scale, in seconds), accuracy of PSVT related questions (scale, in % of correct answers) & behavioral intention (Likert scale responses to 20 questions for both respective systems) measured through technology acceptance model.
  53. 53. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 54 10.3 Research Questions Stated formally, we formed the following research questions: RQ1: Would there be any differences in content understanding, learnability and application between A.R based E Learning tools versus Web based E Learning tools? RQ2: Would there be any effect of A.R and Web based E Learning systems in PSVT performance as compared to mental rotation alone? RQ3: Would the accuracy in PSVT vary with platforms ? (A.R and Web) RQ4: Would there be any differences in completion time for PSVT between Web based systems and A.R based E learning systems? RQ5: Is there any difference in behavioral intention in terms of using the system between web and A.R? RQ6: Would there be any correlation between the learning styles of students (VARK) and their PSVT performance as well as their solid state scores?
  54. 54. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 55 Figure 40: Participants filling Pre Questionnaire 10.4 Participants We recruited forty participants for our study compromising mostly of first year engineering students who had studied the topic of our case-study, i.e. Solid State chemistry in the past one year. We conducted an initial phase of questionnaire based survey with these participants. This questionnaire consisted of 3 parts: 1) The VARK questionnaire 2) Solid State chemistry related questions 3) PSVT:R The VARK questionnaire was used to provide us with insights into the learning styles of our participants which could be used in later stages to draw some inferences. The second part of the questionnaire involved 6 concept based solid state questions to test the current understanding and remembrance of these concepts in the participants. The last part was the standardized PSVT conducted in a stipulated time limit of 15 minutes to gauze the spatial ability of our participants. The PSVT test was conducted on paper. The forty participants were then divided into two groups based on their PSVT and solid state scores, such that the average distribution of both the scores is same in both the groups. These two groups were then used for our comparative analysis experiment wherein one group used a web based e learning system whereas the other group used an A.R based system. Study duration varied per participant, due to differing reaction times, but on average participants took around an hour, with 10 minutes to explore the system, another 10 minutes for breaks and questionnaires and 40 minutes for PSVT. Participants were not paid for involvement.
  55. 55. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 56 10.5 Set up & materials We implemented our A.R system on an android tablet device as an application and used standard XII chemistry NCERT textbook as the image tracker. The application was build using vuforia sdk on unity platform. The Web system was built using bootstrap framework with sketch fab plugin used to embed 3D models. The 3D models used in both the web system and A.R system were built using sketchup. Questionnaires: Two Solid State content related quizzes (before and after using tool) were developed which was later validated by chemistry teacher. First quiz which was given before the main study, had basic and fundamental questions of the chapter to just gauge their current retention of the chapter knowledge. Second quiz given to students just after using the tool, had conceptual, visualization based questions which were related the content (octahedral and tetrahedral voids) shown to them while using the tool. We used the online service of google forms to record responses for content related questions as well as for recording responses for technology acceptance model. We used 20 items sheet of Purdue visualization test of rotation (PSVT:R) by [Bodner and Guy, 1997]. For having responses of PSVT, we used an online service proprofs (www.proprofs.com) which recorded the time taken by each participant for each of the questions in the background. In TAM questionnaire, we had 3 questions each on Perceived Ease of Use, Perceived Usefulness and Attitude from [Davis, 1989], 3 questions on behavioural intention from [Davis, Bagozzi et. al., 1989], three on computer-self efficacy from [Compeau et.al. ,1995], three on perceived enjoyment adapted from [Moon & Kim, 2001]. All questionnaires are included in the appendix section. Each trial was conducted in a quiet lab environment. .
  56. 56. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 57 Figure 41: Experiment with A.R (up) & Web based system (below) 10.6 Procedure The comparative analysis experiment was conducted with 40 participants wherein 20 participants were given an A.R system and the other 20, the web system. This was done about a week after the pre questionnaire was filled, so that there was sufficient time gap between the participant’s attempts at PSVT. All the sessions were video recorded with prior permission from participants. At the start of the trial, participants were given a hand-out describing different sections of the study and the task for each. Having read the instructions, the participants were asked if they had understood how the test would proceed, and any questions that arose were answered. The study began with a demonstration / walkthrough of the system where different interactions of the system were shown and verbally explained. The participants were told to pay focus on the content since the subsequent questionnaire involved questions related to conceptual understanding of the content showcased. The users were also given assurance that there would not be any memory based questions asked and that they should only focus on understanding and learning of the 3D concepts rather than remembering them. The participants were then given the system for free exploration and content viewing for as much time as they needed. Headphones were provided with the tablet to the A.R users and with the laptop to the Web system users for audio content. Once the users indicated they had completed viewing the content, the system was taken and they were asked to answer an online questionnaire which contained 8 conceptual questions related to the concepts shown in the system. Rough sheet and a pen were provided to the users. This questionnaire also did not have a time limit - users had the freedom to take as long as they wanted. Once completed, qualitative feedback about the system and the questions was taken. The demo, free exploration and solid state questionnaire was followed by a five minute break where we offered chocolates to our participants. Post the mid session break, the users began with the PSVT test. Since the users had already taken the PSVT as a part of the pre questionnaire, they were familiar with the format and types of questions.
  57. 57. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 58 Figure 42: Experiment with A.R (up) & Web based system (below) This time though, there were 3 changes in conducting PSVT, as explained below- 1. Users were given the system (A.R application on a tablet device to 20 users and Web to 20) to help them answer these questions. Both these systems contained the 3D models asked in PSVT which could be rotated and viewed from all angles in 3 dimension. Users were expected to use these systems as a help in answering. 2. The PSVT was conducted online through proprofs instead of on paper like the previous time. This helped in tracking the time taken by each participant to answer each of the twenty questions in the background. 3. There was no time limit given to to users (unlike last time) since they were using a system to help them answer and getting familiar with its use and application is expected to take varying time with users. The users were allowed to take small breaks between questions if required. This was implemented by adding a “are you ready to proceed?” question in the questionnaire before each question. Users were instructed that only when they are ready to answer the next question should they proceed and that they could take breaks in between at such questions. Once the PSVT was complete, qualitative feedback regarding the system and questionnaire was again taken. The final session consisted of the TAM questionnaire which was given to the users through an online Google form. The users were instructed to answer each of these questions independently without any overall biasness.
  58. 58. Final Year Design Project – Studies in application of augmented reality in E Learning Courses Cle ar n /////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////// Department of Design, IIT Guwahati 59 Mean SD Solid State AR 4.2 1.15 Solid State Web 4.25 1.29 PSVT AR 14.7 3.08 PSVT Web 14.45 3.58 Figure 43: Pre Questionnaire mean & standard deviation Min Max Mean SD Visual 1 13 5.63 3.14 Auditory 2 14 7.15 2.77 Reading 2 12 5.65 2.33 Kinesthetic 2 13 7.6 2.59 Total 16 43 26.03 6.98 Figure 44: Pre Questionnaire VARK mean & standard deviation Chapter 11 – Results 11.1 Quantitative Pre-questionnaire results Before main study, we collected Solid state content quiz scores and Purdue Visualization Test scores from forty students to measure their content retention and spatial ability and divide them into two similar groups of twenty each. We also gathered VARK learning style scores for each participant to explore relationship between learning style and other test performances. Means and Standard Deviation of content quiz score and PSVT score for the two groups are mentioned in the figure. Mean values of solid state quiz scores of all forty was 4.23 and of PSVT scores was 14.58. Dominance of learning style was found using mean and standard deviation values of individual learning mode which are mentioned in the table (). From the table, it is clear that kinesthetic was most strong learning mode in these participant followed by auditory whereas Visual was least dominant mode. Main study results After both of the groups had been subjected to use two different e-learning tools, new solid state quiz score, PSVT accuracy score, individual PSVT question response time, total PSVT completion time, Technology Acceptance Model responses for each participant were collected and analyzed. An independent t-test was Solid-state accuracy An eight questions quiz related the content shown during tool usage, was given to the participants to measure their ability to understand and apply the concepts after using the tool. Content accuracy score was calculated as the number of correctly answered questions (out of eight). From Shapiro-Wilk Test and skewness-kurtosis analysis, it was confirmed that solid state score distribution is non-parametric. Therefore, due to non-parametric nature of data and independent sample design, Mann- Whitney U test was used to determine

×