Are Sensory EffectsReady for the World Wide Web? <br />Christian Timmerer, Markus Waltl, and Hermann Hellwagner<br />Klage...
Outline<br />Background / Introduction<br />MPEG-V Media Context and Control<br />Concept of MPEG-V Sensory Information<br...
MPEG-V: Media Context and Control (MCC)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />3<br ...
Concept of MPEG-V Sensory Information<br />Consumption of multimedia content may stimulate also other senses<br />Vision o...
Sensory Effect Description Language (SEDL)<br />XML Schema-based language for describing sensory effects<br />Basic buildi...
Sensory Effect Description Language (cont’d)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />...
SEVino, SESim, SEMP, and amBX<br />amBX (Ambient Experience) system + SDK<br />Two fan devices, a wrist rumbler, two sound...
Are Sensory Effects Ready for the World Wide Web?<br />How to embed SEM into Web documents?<br />How to interpret/parse SE...
Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />Synchronization with audio/video (cont’d)<br />currentTime...
Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />How to extract color information automatically?<br />Used ...
Conclusions<br />Introduction to MPEG-V Sensory Effects<br />Are Sensory Effects Ready for the World Wide Web? – Yes, they...
Thank you for your attention<br />... questions, comments, etc. are welcome …<br />Ass.-Prof. Dipl.-Ing. Dr. Christian Tim...
Upcoming SlideShare
Loading in...5
×

Are Sensory Effects ready for the World Wide Web?

1,771

Published on

Published in: Technology, Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,771
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Currently, the following media types are defined: ‘aural’, ‘braille’, ‘handheld’, ‘print’, ‘projection’, ‘screen’, ‘tty’, ‘tv’
  • Are Sensory Effects ready for the World Wide Web?

    1. 1. Are Sensory EffectsReady for the World Wide Web? <br />Christian Timmerer, Markus Waltl, and Hermann Hellwagner<br />Klagenfurt University (UNIKLU)  Faculty of Technical Sciences (TEWI)<br />Department of Information Technology (ITEC)  Multimedia Communication (MMC)<br />http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.at<br />20 May 2010<br />Acknowledgments. This work was supported in part by the European Commission in the context of the NoE INTERMEDIA (NoE 038419), the P2P-Next project (FP7-ICT-216217), and the ALICANTE project (FP7-ICT-248652). <br />
    2. 2. Outline<br />Background / Introduction<br />MPEG-V Media Context and Control<br />Concept of MPEG-V Sensory Information<br />Sensory Effect Description Language (SEDL)<br />Software/Hardware components: SEVino, SESim, SEMP, and amBX+SDK<br />Are Sensory Effects Ready for the World Wide Web? <br />How to embed SEM into Web documents?<br />How to interpret SEM within a browser?<br />How to synchronize with audio/video content of the Web document?<br />How to extract color information automatically?<br />Conclusions and Future Work<br />2010/05/20<br />2<br />Christian Timmerer, Klagenfurt University, Austria<br />
    3. 3. MPEG-V: Media Context and Control (MCC)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />3<br />System Architecture<br />Pt. 1: Architecture<br />Pt. 2: Control Information<br />Pt. 3: Sensory Information<br />Pt. 4: Virtual World Object Characteristics<br />Pt. 5: Data Formats for Interaction Devices<br />Pt. 6: Common Types and Tools<br />Pt. 7: Conformance and Reference Software<br />http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-V<br />
    4. 4. Concept of MPEG-V Sensory Information<br />Consumption of multimedia content may stimulate also other senses<br />Vision or audition<br />Olfaction, mechanoreception, equilibrioception, thermoception, …<br />Annotation with metadata providing so-called sensory effects that steer appropriate devices capable of rendering these effects<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />4<br />… giving her/him the sensation of being part of the particular media<br />➪ worthwhile, informative user experience <br />
    5. 5. Sensory Effect Description Language (SEDL)<br />XML Schema-based language for describing sensory effects<br />Basic building blocks to describe, e.g., light, wind, fog, vibration, scent<br />MPEG-V Part 3, Sensory Information<br />Adopted MPEG-21 DIA tools for adding time information (synchronization)<br />Actual effects are not part of SEDL but defined within the Sensory Effect Vocabulary (SEV)<br />Extensibility: additional effects can be added easily w/o affecting SEDL<br />Flexibility: each application domain may define its own sensory effects<br />Description conforming to SEDL :== Sensory Effect Metadata (SEM)<br />May be associated to any kind of multimedia content (e.g., movies, music, Web sites, games)<br />Steer sensory devices like fans, vibration chairs, lamps, etc. via an appropriate mediation device<br />➪ Increase the experience of the user<br />➪ Worthwhile, informative user experience<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />5<br />
    6. 6. Sensory Effect Description Language (cont’d)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />6<br />SEM ::=[DescriptionMetadata](Declarations|GroupOfEffects|Effect|ReferenceEffect)+<br />Declarations ::= (GroupOfEffects|Effect|Parameter)+<br />GroupOfEffects ::= timestamp EffectDefinitionEffectDefinition (EffectDefinition)*<br />Effect ::= timestamp EffectDefinition<br />EffectDefinition ::= [activate][duration][fade][alt] [priority][intensity][position]<br />[adaptability]<br />
    7. 7. SEVino, SESim, SEMP, and amBX<br />amBX (Ambient Experience) system + SDK<br />Two fan devices, a wrist rumbler, two sound speakers, a subwoofer, two lights, and a wall washer<br />Everything controlled by SEM descriptionsexcept light effect<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />7<br />Annotation Tool: SEVino<br />Simulator: SESim<br />Player: SEMP<br />
    8. 8. Are Sensory Effects Ready for the World Wide Web?<br />How to embed SEM into Web documents?<br />How to interpret/parse SEM within a browser?<br />Directly via JavaScript<br />Browser plug-in <br />How to synchronize with audio/video content of the Web document?<br />HTML5’s video and audio elements and corresponding DOM attributes<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />8<br /><link rel="alternate" type="text/xml”<br />href="http://server.org/sem.xml”media="light, wind, vibration"/><br />
    9. 9. Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />Synchronization with audio/video (cont’d)<br />currentTimeDOM attribute: current playback position in seconds<br />timeupdateevent: indicates that the current playback position changed as part of normal playback <br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />9<br />
    10. 10. Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />How to extract color information automatically?<br />Used for controlling ambient lights<br />currentFramedoes neither exist nor is foreseen in HTML5<br />canvas element allows for manipulating the display’s content<br />(+) Pro: extract the pixel information from the display<br />(-) Con: needs to be added explicitly during authoring<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />10<br />
    11. 11. Conclusions<br />Introduction to MPEG-V Sensory Effects<br />Are Sensory Effects Ready for the World Wide Web? – Yes, they are!<br />Future work<br />Implementation is ongoing<br />Subjective quality assessments<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />11<br />
    12. 12. Thank you for your attention<br />... questions, comments, etc. are welcome …<br />Ass.-Prof. Dipl.-Ing. Dr. Christian Timmerer<br />Klagenfurt University, Department of Information Technology (ITEC)<br />Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA<br />christian.timmerer@itec.uni-klu.ac.at<br />http://research.timmerer.com/<br />Tel: +43/463/2700 3621 Fax: +43/463/2700 3699<br />© Copyright: Christian Timmerer<br />12<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×