Uploaded byAnasol Peña-Rios
Towards blended reality on collaborative laboratory activities using smart objects
The document discusses the integration of blended reality in collaborative laboratory activities at the University of Essex, focusing on the use of smart objects to enhance distance learning. It outlines the challenges faced in current educational technologies and proposes a mixed reality environment to facilitate interactive and immersive learning experiences for students across geographical locations. The implementation of an interreality portal aims to combine real and virtual elements to create a seamless educational experience.
Towards blended reality on collaborative laboratory activities using smart objects
- 1. Towards blended realityon collaborative laboratory activities using smart objects School of Computer Science & Electronic Engineering University of Essex, UK Anasol Peñ a-Ríos acpena@essex.ac.uk 1
- 2. • Introduction • Blendedlearning landscapes • Scenario: Collaborative laboratory activities • Interreality environment (Conceptual model & implementation) • Blended reality distributed system • Collaboration projects • Summary Overview of the presentation 2
- 3.
- 4. Introduction The Economist IntelligenceUnit, “The future of higher education: How technology will shape learning,” The Economist, 2008. 6
- 5. Use of technologyin academic settings (Some challenges) Promote adoption of new technologiesadoption of new technologies by academic faculty members and students Adapt technologies, pedagogicalAdapt technologies, pedagogical methods and evaluationsmethods and evaluations enabling this integrated solutions for people in geographically dispersedgeographically dispersed locations Change the way to experienceexperience education from one-dimensionaleducation from one-dimensional (physical) to multi-dimensionalmulti-dimensional (physical and virtual) education in an integrated way BlendedBlended LearningLearning LandscapesLandscapes 7
- 6. Blended learning landscapes Virtual Learning Environmen t Blended Learning Environment RealLearning Environment An infrastructure that works as a link between this environments able to reflect information (data, changes, etc) on real time Learning activities that complements virtual and real world making possible for students and tutors to move between these two in a seamless way 8
- 7. Blended Reality This might belimited by the capacity of user’s presence and engagement to a single reality at a time (Lifton's "Vacancy problem"1) a consequence of the user’s real immersion. 9 User's interaction in real- time with two different environments (real/local and virtual/distant) extend them to work as if they were one, by blending traces of one into the other unconsciously (often seemingly as simultaneously). 1. J. Lifton and J. Paradiso, “Dual Reality: Merging the Real and Virtual,” Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 33, no. 1, pp. 12-28, 2010
- 8. Scenario (Work inprogress) COLLABORATIVE LABORATORY ACTIVITIES 10
- 9. Laboratory activities fordistance learners Which are the options for remote laboratory activities? • Simulations • Virtual laboratories • Remote laboratories • To test conceptual knowledge • To work collaboratively • To interact with equipment • To performing analysis on experimental data Laboratory activities 11
- 10. Examples University of Bristol- LabSkills University of Leeds - Virtual Labs Challenges for current technology: •Limited interaction with real equipment •Use of idealised datasets •Restricted collaborative interaction Freie Universitat Berlin – Universit Technology Enhanced Textbook 12
- 11. Interreality Environment 1 -J. van Kokswijk, Hum@n: Telecoms and Internet as Interface to Interreality : a Search for Adaptive Technology and Defining Users, Bergboek, 2003 A collection of interrelated devices in the real world, a virtual environment and software agents that allow users to complete activities at any point of Milgram’s Virtuality Continuum to achieve user perception of integration between the physical and the virtual world. 13 Interreality.- “a hybrid total experience between reality and virtuality”1
- 12. Interreality Portal (Definition) Inthe educational context can be used: • As a learning environment • For geographically dispersed students • Using mixed reality laboratory activities 14 Human-computer interface (HCI) which captures data obtained in real-time, and processes this data so it can be mirrored in the virtual world linking both worlds.
- 13.
- 14. A "real" immersive environment •Givesto the learners the sensation of "being there" A 3D virtual environment •3D GUI made on Unity3D, a cross-platform game engine used to create interactive 3D content. xReality objects 1- Immersive Display Group (www.immersivedisplay.co.uk) 2 - V. Callaghan, «Tales From a Pod» on Creative Science Workshop, Nothingham, 2010. 1, 2 ImmersaStation Interreality Portal (implementation) 16
- 15. 17 Smart objects • coupledto their virtual representation (dual reality state) • updated and maintained in real time 1.- Kortuem, F. Kawsar, D. Fitton and V. Sundramoorthy, “Smart objects as building blocks for the Internet of things,” IEEE Internet Computing, vol. 14, no. 1, pp. 44-51, 2010 Autonomous physical/digital objects augmented with sensing, processing, and network capabilities” which can interpret their local situation and status, and can communicate with other smart objects and interact with human users 1 +Xreality objects =Smart objects
- 16. Interaction between physicaland virtual elements within an environment • Each object, (real and virtual), is complete by itself and can exist without the other. • Dual reality state enriches both objects by the bi-directional process that can reflect, influence and merge real-time information. MIT Media Laboratory – Dual reality lab 2008 18 CrossReality (xReality)
- 17. xReality objects &virtual objects • Real world objects that have a virtual representation • Dual reality state xReality objects Virtual objects • Only exist inside a virtual environment • Rules and behaviours but these depend on the existence of the virtual world. 19 Anasol Peñ a-Ríos, V. Callaghan, M. Gardner & M. J. Alhaddad, «Remote mixed reality collaborative laboratory activities: Learning activities within the InterReality Portal»on Web Intelligence and Intelligent Agent Technology (WI-IAT), 2012 IEEE/WIC/ACM International Conferences - The Intelligent Campus International Symposium (IC’12), vol.3, no., pp.362,366, 4-7, Macau, China, 2012.
- 18. xReality objects (Implementation) *Fortito (www.fortito.mx) 20 **Raspberry Pi Foundation (www.raspberrypi.org) Main module Components • Fortito’s* Buzz-Board Educational Toolkit • 30 pluggable boards that can interconnect and communicate between each others via I2C protocol • Comprises different sensors and actuators to allow the creation of diverse physical mashups • Raspberry Pi ** • Detects other components and works as a hub to connect them to the interreality system
- 19. Interreality Portal •Buzz boards http://youtu.be/wblqY-M0uKA?t=38s •InterrealityPortal http://youtu.be/3oqdr7 nIG7Y http://youtu.be/4AZw_ gcbAzw?hd=1 21
- 20. Implementation (architecture) (singledual reality) 22 Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Developing xReality objects for mixed- reality environments»on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.
- 21. Implementation (single dualreality) 23 Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Developing xReality objects for mixed- reality environments»on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.
- 22. Multiple dual realities 24 •Management of multiple dual reality states • Addition of a collaborative layer to allow two or more learners in different geographical locations participate on the laboratory activity +Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «End-user programming & deconstrutionalism for collaborative mixed reality laboratory co-creative activities»on 2nd European Immersive Education Summit (EiED’12), Paris, France, 2012.
- 23. Blended reality distributedsystem (Conceptual model) 25 Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Developing xReality objects for mixed- reality environments»on 1st Workshop on Cloud of Things (CoT'13), Athens, Greece, 2013. Unplublished conference paper.
- 24. End-user programming The interactionbetween users within collaborative mixed reality learning activities can be analysed from two different angles: •From the learners view •From the instructors view. The learning environment should allow users to create and execute learning activities regardless their expertise on computers. 26
- 25. • Homogeneity &structure to the creation of mixed reality learning activities • Adding collaborative learning to the use of xReality objects Learner’ s view Instructor’ s view 3D Learning environmen t End-user programming 27
- 26. Learning Activities Learning activitieswithin the InterReality Portal are structured as a sequence of activities based on the IMS Learning Design specification (Units of Learning-UoL). CollaborativeCollaborative interactioninteraction betweenbetween learnerslearners EmphasiseEmphasise ComputingComputing fundamentalsfundamentals Problem-Problem- basedbased learninglearning VirtualVirtual objectsobjects (SW)(SW) Internet-of-Internet-of- ThingsThings applicationsapplications xRealityxReality objectsobjects (HW)(HW) ComputerComputer sciencescience projectsprojects 28
- 27. Deconstructed model • Basedon the disaggregation of physical/logical devices and services. • Propose the creation/identification of a number of elementary services (atomic functions) which can be combined in various ways to create complex functions (nuclear functions). 29
- 28. Deconstructionism Role Atomic functionNuclear function Learner Objects available in the environment, actions available (programming statements). An Internet-of-Things project Instructor Resources available in the environment, activities available (sequence of activities). A Unit of Learning (UoL) Technical Infrastructure Processes, threads, processors or FPGA xReality toolkit and system 30 Anasol Peñ a-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «End-user programming & deconstrutionalism for collaborative mixed reality laboratory co-creative activities»on 2nd European Immersive Education Summit (EiED’12), Paris, France, 2012.
- 29. Implementation phases 31 Anasol Peña-Rios, V. Callaghan, M. Gardner & M. J. Alhaddad, «Towards the Next Generation of Learning Environments: An InterReality Learning Portal and Model»on 8th International Conference on Intelligent Environments 2012 (IE'12), Guanajuato, Mexico, 2012.
- 30. Scale Up project •A collaboration between King Abdulaziz University, KSA and the University of Essex,UK. 32
- 31. Summary • Core elementsof our mixed reality activities • Combine them using end-user programming to create an educational mixed reality object • Immersive mixed reality learning environment • Grounded on collaborative, constructionalist, problem-based learning theories • Offers the possibility of creating collaborative laboratory activities for geographically dispersed students InterReality Portal xReality & Virtual objects 34
- 32. Challenges Pedagogical challenge: aconstructionist student-centred method to create laboratory activities for distance learners • A representation of a deconstructed worldTechnical challenges • Learners • Instructors • A set of components that can be shared and combined • Distribution of objects between different immersive environments • Blended reality distributed system 35
- 33. Challenges (2) • Tosolve this we have proposed the use of a deconstructionist architecture. Physical model of distributed xReality objects in an immersive learning environment Pedagogical model of constructionist laboratory activities A solution forA solution for distributeddistributed mixed realitymixed reality laboratorieslaboratories 36
- 34. Questions •Thank you foryour attention! acpena@essex.ac.uk acpena@ieee.org 37
Editor's Notes
- #4 Me à PhD Candidate and Information Officer at the Immersive Education initiative European chapter. My research focused on Intelligent Learning Environments, mixed reality and immersive education. This research is part of a collaborative project between Essex and King Abdulaziz Uni to scale-up existing technology in intelligent environments, from smaller facilities such as the University of Essex iSpace to a complete campus with multi-story large footprint buildings.
- #7 Survey conducted by The Economist magazine in 2008 with 289 executives responded the survey, 189 participants came from higher education and 100 came from corporate settings
- #9 A virtual learning environment (web/ 3D world) A physical learning environment (real world)
- #10 ejemplo Blended Reality: use of videoconferences (Skype) ejemplo Vacancy problem: everyday activities as run
- #12 But what happen with laboratory activities?
- #13 University of California Davis School of Medicine - Virtual Hallucinations Project: http://www.ucdmc.ucdavis.edu/welcome/features/20070404_virtual_psych/index.html Video (a bit disturbing): http://www.youtube.com/watch?v=s33Y5nI5Wbc Sakshat Amrita Virtual Lab (Amrita University) Project: http://amrita.vlab.co.in/ Video: http://www.youtube.com/watch?v=ViqHtlZSOjM Ohio University College of Osteopathic Medicine - Virtual Haptic Back Project for palpatory training http://www.ohio.edu/people/williar4/html/VHB/VHB.html University of Bristol - LabSkills http://www.bristol.ac.uk/news/2009/6154.html Software was developed by Bristol Uni and currently it is used in Durham Uni and Notthingham Uni (each uni gave a talk about the use of this software in their courses) http://www.labskills.co.uk/ University of Leeds - Virtual Labs http://www.virtual-labs.leeds.ac.uk/pres/index.php Freie Universitat Berlin - UniversitTechnology Enhanced Textbook (TET) http://didaktik.physik.fu-berlin.de/projekte/tet/ http://didaktik.physik.fu-berlin.de/IMPAL/show/demo.php Durham University - Interactive screen experiments - physics http://level1.physics.dur.ac.uk/general/index.php
- #15 The interaction between the elements can be defined as Cross-Reality (xReality) or dual reality
- #16 The implementation of the InterReality Portal is based on three major components: A real environment A 3d virtual environment xReality objects and Virtual objects
- #17 we combine 1) a semi-spherical sectioned screen that includes a desk allowing the user to be sit in a natural position to perform learning activities, with a free-range of head movement without the need of any intrusive body instrumentation, based on a specification from the sci-fi story “Tales from a Pod” 2) a camera that allows students to interact between them and the environment, 3) a network of sensors and actuators to obtain real-time information for object identification and replication for dual reality states.
- #20 Following the ideas of smart objects on the Internet-of-Things (IoT), an xReality object has a unique ID, a list of available services (e.g. to get data or receive data) and in some cases rules (e.g. a certain object cannot work without fulfilling some preconditions). In a similar way each virtual object has a unique ID, one or more behaviours attached (e.g. the virtual object must behave as a solid object according to physical variables, such as weight, gravity, etc.) and rules similarly to the xReality object
- #21 formed by a group of interchangeable pluggable components, to create a mashup with a main component which identifies and integrates the others
- #25 As long as the session continues, changes in any of the objects will be managed by the Context-Awareness agent and the Mixed Reality agent considering the following scenarios: A change in any Virtual object of a given InterReality Portal results in identical changes to all subscribing InterReality portals. A change in an xReality object of a given InterReality Portal results in changes in the representation of the real device on all subscribing InterReality portals.
- #27 Learners view: During the learning session when they interact with the environment and between them, they are programming a series of actions to be executed by the objects. Instructors view: While they are creating a UoL, they are establishing a sequence of activities to be performed by the learners during the educational session.
- #29 is to built computer science projects combining hardware (xReality objects) and software modules (virtual objects) to create Internet-of-Things applications emphasising computing fundamentals grounded on co-creative and collaborative interaction between learners using problem-based learning (PBL). A diversity of resources may be available to support learning however this process cannot be complete without… IMS = Instructional Management Systems (IMS) Global Learning Consortium BENEFITS à portability and reusability Problem-based learning is a constructionist method that allow students to construct their own knowledge by the correlation between concepts and proposed solutions to real world problems performed in realistic settings
- #31 From the instructors’ perspective à the creation of UoL is based on activities (atomic functions) which can be combined to create nuclear functions (a complete UoL). From the learners’ perspective à xReality and virtual objects (atomic functions) can be considered as part of a deconstructed set of components that students can reconstruct in any combination to generate their own unique xReality project – or to reconstruct a xReality project that is prescribed by the instructor – (nuclear functions). From a technical (system architecture viewpoint) à the deconstructed elements become sets of autonomous networked resources, that may be inter-connected to form different combinations (constructed) forming a variety of student projects or UoL as required by the teacher or student.
- #32 our research is moving from phase 1 to phase 2, integrating the InterReality Portal implementation with the end-user programming concepts discussed within this paper. The possibility of combining virtual and xReality objects with deconstructionism and the use of end-user programming to create mixed reality learning activities that can be constructed and shared by teams of geographically dispersed students is our final goal.
- #35 We presented
- #36 We presented a series of challenges for constructing this model