This document discusses mixed reality learning games (MRLG) that merge real and virtual worlds. It proposes using augmented reality and augmented virtuality to enhance learning through interactive objects and user actions. A framework is presented that uses multiagent and layered architectures to design, generate, and implement MRLGs based on a model-based approach. Examples of MRLG software are provided to motivate learners through methods from video games.

1. Mixed Reality Learning Games (MRLG)
UMR 5205
Florent Delomier, Bertrand David, René Chalon
Laboratoire d'InfoRmatique en Image et Systèmes d'information - SILEX Team
UMR5205 CNRS/INSA de Lyon/Université Claude Bernard Lyon 1/Université Lumière Lyon 2/Ecole Centrale de Lyon,
Ecole Centrale de Lyon, 36 avenue Guy de Collongue, Ecully-Lyon, 69134, France
http://liris.cnrs.fr/membres?idn=fdelomie
Tel: +33 4 72 18 61 48..; fax: +33 4 72 18 61 52 ; e-mail: florent.delomier, bertrand.david and rene.chalon @ec-lyon.fr
Objectives
Provide mixed reality (MR) design needs Proposal of different embodiment of interactive objects and
Provide methodology and framework to include MR in actions into potential mixed objects and user actions
Learning Game (LG) with presentation and action patterns Influence on the software and hardware architecture design
Mixed reality refers to the merging of real and virtual worlds to produce new environments and visualizations where physical and digital
objects co-exist and interact in real time. Learning games are computer based pedagogical tools that used method outcome from video
games to motivate learners to realize some learning activities. The aim is to help to use both.
A Mixed Reality context Representations and Actions Patterns according to
embodiments features of interactive objects and user actions
Real env. Digital env.
Augmented Reality (AR) Augmented virtuality (AV)
For each interactive objects
AR allows to link digital data to physical AV allows to increase interactivity with digital
objects object by physical object uses
Interactive object’s representation perception
Exemple of pedagogical
Use of MR env. in educational tools software in MR env. to
enhance learning
Augmented virtuality with
Tacit knowledge a) SharedView
movement,or motiion sensor (unconsciousness)
b) Trainning Tool for assembly
Augmented reality
c) Magic Book
d) Augmented Chemistry Physical representation Mixed reptesentation Digital representation Without representation
m
e) Careta
Technical b Environment related f) SandScape
i a
gestures j t knowledge
l g) Professional training for
c
k maintenance
h) Assembly
Interactive object’s representation meanings
Procedural Declarative i) Virtual Réality Mirror
d n
knowledge knowledge j) VR Post-Stroke Hand
(know how, (know about, Opening Rehabilitation
focused on focused on k) Virtual reality and augmented
o reality in digestive surgery
action) description)
s r n l) Casper
a
Procedure to b m)Distance Learning support
p e Logical reasoning
h d
complete a task g n) Topobo
t
based on Realistic representation Symbolic representation Meaningless representation
f o) FlowBlocks
concepts
p) System Blocks
q) Collaborative Augmented
reality environment
Explicit Knowledge
r) Mixed-reality classroom
s) Karma
For each task different user actions can be chosen
Augmented reality
(consciousness) Tangible interface t) GestureCam
Realistic user action Symbolic user action Meaningless user action Without user action
Use of the object like in the Gesture linked to one or more
(Amalberti, 2001) knowledge distinction No user action asked. The
real use condition (with real knowing about objects and Gesture without link with the
action is triggered when all the
object use) or mime (without tools, their function and the interactive object
conditions are fulfilled
real object use) of this situation way in which they are used
Choosing just one of this type of environment may doesn’t work with different types of learning
objectives at the same time. A more accurate approach is needed.
MRLG software architecture An iterative MRLG software and hardware prototyping
Both use of a multiagent view architecture and layed view architecture in same time to help
design, generation and implementation according to a model based approach.
Semi-concrete User Interface
MRLG specific
Concrete User Interface
Abstract User Interface
component
Tasks and Concepts
Final User Interface
World and
Component
Physics
Adaptor
Domain
Game logic control
Player AI
data
Interactive agent specification with AMF-C Architecture
Paintbrush
Paint
Agent Toy
4 tabletops and 1 tablet as MR devices
Finger detection on the surface
Allow collaboration between
Shape detection on the surface
learners Display digital data on the surface
IR emitting object detection above
Allow users movements between Physical object on the surface
AMF agent World the surface
each tabletops
QR code detection by tablet camera
Intrance Paintline
area area Lea®nIT as MRLG example
Paint
brush
toy Production chain simulation Provide some modification of the Allows the anchor of Lean
with pupils as operator chain to improve it manufacturing knowledge
Concepts model
Tasks model