2011 06 01 (uned) emadrid aelsaddik uottawa tangible objects for e learning

Tangible Objects for E-Learning Abdulmotaleb El Saddik FIEEE, FCAE, FEIC

At a glance Creation of Advanced Multimedia LO Protection of LO Navigation of LOR by VR metaphors Adaptation of LO according to user context/profile & network QoS LO Delivery to user

Introduction
Haptics
Derived from the Greek verb “haptesthai”
Refers to sensing and manipulation through touch
Comprises both “Tactile” and “Kinesthetic” senses
Applications
Education and Training
Medicine: Tele-surgery, rehabilitation, etc.
Entertainment
Arts and Design
…

Multimedia and Authoring
Multimedia contents
Graphic images
3D models
Audio and video files
And recently haptic stimuli
Multimedia authoring tools
Integrate the disparate media elements into a cohesive multimedia application

Multimedia Authoring Tools
A major challenges in multimedia authoring is to make authoring complex multimedia titles as easy as using a word processor or drawing program
We propose a multimedia authoring tool for hapto-visual applications to make the development process a non-programming experience

HAMLAT
Blender:
Open source
Full-fledged 3D graphical renderer
Physics and game engine
Adaptive user-interface
Plug-ins incorporated in Blender source code
Haptic panel in Blender GUI
Haptic rendering extension
Implemented in C++ using OpenHaptics and OpenGL

HAMLAT Implementation (demo)

A Multimedia Handwriting Learning and Evaluation Tool
Learning sensorimotor skills is difficult with graphical or auditory feedback
Sense of touch will be a must for physical guidance
Examples of physical guidance learning:
Handwriting for visually impaired people
Medical procedures and rehabilitation training
Painting/sculpting techniques

Multimedia Learning Tool
Distinguished features
Language choice: Arabic, French, English, Japanese and Spanish
Multimodal: haptic, visual, and audio information for each letter
Learning mode choice: full or partial guidance, or test mode.
Evaluation Mode: quantitative evaluation using dynamic time warping

Multimedia Learning Tool

Learning Mode -1-
Language Repository
Including symbols images, pronunciation files, and the haptic stimuli of each letter (using XML-based schema)

Learning Mode -2-
Haptic player: Three modes of operation:
No guidance
Partial guidance
Full guidance

Evaluation Mode -1-
Test GUI

Evaluation Mode -2-
Dynamic Time Warping
Measure the similarity between test and reference handwritings
Tolerate both global and local shifts in the time domain
Startup position of the test pattern is shifted to that of the reference pattern
Given two time series // x i is a point in the test trajectory X = x 0 , x 1 , …, x N-1 // y i is a point in the ref. trajectory Y = y 0 , y 1 , …, y T-1 D(x i ,y i ) = sqrt((abs(x i )-abs(y i ))^2 + (ord(x i )-ord(y i ))^2) DTW[0,0] = D(x 0 ,y 0 ) For i=1 to T-1 DTW[0,i] = D(x 0 ,y i ) + DTW[0,i-1] For i=1 to N-1 DTW[i,0] = D(x i ,y 0 ) + DTW[i-1,0] For i=1 to N-1 For j=1 to T-1 DTW[i,j]=D(x i ,y j )+min{DTW[i,j-1],DTW[i-1,j],DTW[i-1,j-1]} Return (DTW[N-1,T-1])

Performance Evaluation -1-
Experimental Setup
The Phantom Omni Device
Application is developed with .NET 2.0 Framework and programmed using C#.
Bipolar grading scheme (Pass/fail)
Threshold distance is computed based on subjective analysis
Three threshold values were estimated:
Difficult
Medium
Easy

The DTW distance for the “Tha” Arabic character

The DTW distance for the “i” Japanese character

Performance Evaluation -4- (Demo) The DTW distances for five Japanese characters Character Threshold Distance (x10 3 mm) Easy Medium Difficult Sa 5.5 2.5 1 Shi 4 1 0.5 Chi 5 2.5 1 Tsu 4 1.5 1 Na 10 6 1

At a glance Creation of Advanced Multimedia LO Protection of LO Navigation of LOR by VR metaphors Adaptation of LO according to user context/profile & network QoS LO Delivery to user

Haptic signal prediction, compression, and watermarking Watermark haptic signals to protect copyright and authenticate content.

Case Study: Identifying Human Pattern with Haptics The Experiment Methodology Dynamic Time Warping : + Nelder-Mead non-linear minimization Spectral analysis: Fast Fourier Transform Unsupervised Method: K-Means Graphic Representation Data Trial Timestamp Position X Position Y … Force X ( N) 1 0 0.23344 0.56768 0.00456 1 … .. … … 1 0.0123090 0.37676 0.98976 0,03767

Verifying such feasibility ( demo ) Virtual Check Virtual Mobile Phone Performance Ambient Intelligent Engine

At a glance Creation of Advanced Multimedia LO Protection of LO by Digital Watermarking Navigation of LOR by VR metaphors Adaptation of LO according to user context/profile & network QoS LO Delivery to user

Navigation
Tangible Objects: Demo1
Haptics: Demo 2
Mobile Phone: Demo 3

E-Learning courses
Toddler Interfaces

Toddler Interfaces
Main idea
Whenever a child utters an object’s name or taps on an object, a set of appropriate multimedia representations is displayed (i.e. images, Multi-lingual texts, audio…)
Techniques investigated
Voice-based technique (Speech detection)
Magic Stick technique (RFID and Bluetooth technologies)
Evaluation
Several games were developed: i.e., Book game, Alphabet Learning, and Object Identification

Alphabet Learning Evaluation
12 children, ages 4-5, evaluated the system
We Observed the following:
Children’s discussions
Their Writings
Their Questions
Simplicity of using the Magic Stick.
Demo 1 Demo 2

Clustering and 3D Mapping Semantic keyword metrics MST tree Overall clustering process design

Searching/Sharing Learning Objects

Sample Interface: Dynamic Traffic Layout

Multimodal Interaction ( demo )

Last thoughts

Acknowledgment

October 4, 2005 Dankie WAD MAHAD SAN TAHAY GADDA GUEY Ευχαριστώ Urakoze Asante متشکرم DMnvwd go raibh maith agaibh

2011 06 01 (uned) emadrid aelsaddik uottawa tangible objects for e learning

by emadrid network

Accessibility

Categories

Upload Details

Usage Rights

Statistics