Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010

ICIW 2010 - The Fifth International Conference on Internet and Web Applications and Services
May 9 - 15, 2010 - Barcelona, Spain
Titol de la presentació
Multimodal Interaction in Distributed
and Ubiquitous Computing
Marc Pous and Luigi Ceccaroni
bdigital
Cognom Nom, CàrrecNom de la jornada, lloc

SUMMARY
Why multimodality?
Multimodal distributed services deployment
User tests and conclusions

In our cities, urban services practically have not changed in a century

MOTIVATIONS
1: Location-based services

http://www.flickr.com/photos/nnova/4512346900/

http://picasaweb.google.com/marc.pous/Xina2007#5102695479944027890

http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415374502175450370

http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415371534810675298

http://www.flickr.com/photos/loquat73/3385335980/

MOTIVATIONS
2: Interactions and multimodality

http://www.flickr.com/photos/nnova/4423756547/in/set-72157623598200316/

http://www.flickr.com/photos/nnova/4424521014/in/set-72157623598200316/

MOTIVATIONS
3: Accessibility

http://www.dynamiclanguageblog.com/2009/12/technology-grows-for-hard-of-hearing.html

MOTIVATIONS
Whatisan ICD?
(InteractiveCommunityDisplay)

INREDIS
3G = ACCESSIBILITY + UBIQUITY + INTEROPERABILITY
http://www.inredis.es

INREDIS ICD

Management of distributed services that offer the capability of processing and synthesizing multiple modalities of interaction

DISTRIBUTED INTERACTION SERVICES
Voice services
Voice recognition
Via microphone, real-time streaming  converting the voice signal into text
Voice synthesis
Sign language service
Spanish sign language recognition
Via webcam, real-time streaming  converting the image into text
Emotion service
Emotion recognition (voice + image)
Via webcam and microphone, real-time streaming
Avatar service
Emotional avatar
Sign-language speaker avatar

DISTRIBUTED SERVICES
SIGN LANGUAGE SERVICES

DISTRIBUTED SERVICES
VOICE SERVICES

How to consume multimodal distributed services offered by an ICD?

W3C MMI Architecture
W3C MMI Architecture
http://www.w3.org/TR/mmi-arch/

W3C MMI Framework
http://www.w3.org/TR/mmi-framework/

Proposed multimodal interactionframework

ICD ARCHITECTURE
ICD Architecture

ICD DESIGN and INTERFACES
(1) Interactionidentification
(2) Main page
- Mapor avatar
- Commandbuttons
- Content
- Scenariooptions
(3) Contextualizedmapwithservices
(4) Avatar withmap and Street View service

USER–CENTERED DESIGN
User profiles (Personas)
Blind users
Deaf users
Unable to read
People living or visiting a city
Scenarios
Context-aware informative scenario
Emergency scenario

IMPLEMENTATION

USABILITY TESTING
PLI (PeopleLedInnovation) methodology
Preliminar usabilitytestswith a limitednumber of users
Butthe ICD implementationreceivedgoodfeedback!

CONCLUSIONS and FUTURE WORK
Implementation based on the W3C MMI Architecture idea
Platform able to integrate multimodal interactive distributed services and offer interaction in real-time
Web-based applications and device-independent implementation
Improvement of the user experience and accessibility of people with special needs
What are we working on now?
Real-time interaction: delay reduction between interactions
Browser-based apps not being able to easily access device hardware
Synchronous interaction vs. Asynchronous technologies
Improvement of the distributed services orchestration
Enhancement of the mobile usability

MOBILE PHONES?
http://www.slideshare.net/nokiaconversations/younghee-jung-and-jan-chipchase-presentation-for-nokia-design-london-studio-event-london-29th-april-2008

Thankyou!
Moltesgràcies!
¡Muchas gracias!
Marc Pous and Luigi Ceccaroni
mpous@bdigital.org and lceccaroni@bdigital.org
Titol de la presentació
Cognom Nom, CàrrecNom de la jornada, lloc

http://odyssey.calm.unimas.my	77
http://morpheus.calm.unimas.my	17
http://www.slideshare.net	4

Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010

by Marc Pous on May 11, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 98

Statistics

Multimodal Interaction in Distributed and Ubiquitous Computing - ICIW 2010 — Webinar Transcript