Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Are Sensory EffectsReady for the World Wide Web? <br />Christian Timmerer, Markus Waltl, and Hermann Hellwagner<br />Klage...
Outline<br />Background / Introduction<br />MPEG-V Media Context and Control<br />Concept of MPEG-V Sensory Information<br...
MPEG-V: Media Context and Control (MCC)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />3<br ...
Concept of MPEG-V Sensory Information<br />Consumption of multimedia content may stimulate also other senses<br />Vision o...
Sensory Effect Description Language (SEDL)<br />XML Schema-based language for describing sensory effects<br />Basic buildi...
Sensory Effect Description Language (cont’d)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />...
SEVino, SESim, SEMP, and amBX<br />amBX (Ambient Experience) system + SDK<br />Two fan devices, a wrist rumbler, two sound...
Are Sensory Effects Ready for the World Wide Web?<br />How to embed SEM into Web documents?<br />How to interpret/parse SE...
Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />Synchronization with audio/video (cont’d)<br />currentTime...
Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />How to extract color information automatically?<br />Used ...
Conclusions<br />Introduction to MPEG-V Sensory Effects<br />Are Sensory Effects Ready for the World Wide Web? – Yes, they...
Thank you for your attention<br />... questions, comments, etc. are welcome …<br />Ass.-Prof. Dipl.-Ing. Dr. Christian Tim...
Upcoming SlideShare
Loading in …5
×

Are Sensory Effects ready for the World Wide Web?

2,270 views

Published on

Published in: Technology, Health & Medicine
  • Be the first to comment

  • Be the first to like this

Are Sensory Effects ready for the World Wide Web?

  1. 1. Are Sensory EffectsReady for the World Wide Web? <br />Christian Timmerer, Markus Waltl, and Hermann Hellwagner<br />Klagenfurt University (UNIKLU)  Faculty of Technical Sciences (TEWI)<br />Department of Information Technology (ITEC)  Multimedia Communication (MMC)<br />http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.at<br />20 May 2010<br />Acknowledgments. This work was supported in part by the European Commission in the context of the NoE INTERMEDIA (NoE 038419), the P2P-Next project (FP7-ICT-216217), and the ALICANTE project (FP7-ICT-248652). <br />
  2. 2. Outline<br />Background / Introduction<br />MPEG-V Media Context and Control<br />Concept of MPEG-V Sensory Information<br />Sensory Effect Description Language (SEDL)<br />Software/Hardware components: SEVino, SESim, SEMP, and amBX+SDK<br />Are Sensory Effects Ready for the World Wide Web? <br />How to embed SEM into Web documents?<br />How to interpret SEM within a browser?<br />How to synchronize with audio/video content of the Web document?<br />How to extract color information automatically?<br />Conclusions and Future Work<br />2010/05/20<br />2<br />Christian Timmerer, Klagenfurt University, Austria<br />
  3. 3. MPEG-V: Media Context and Control (MCC)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />3<br />System Architecture<br />Pt. 1: Architecture<br />Pt. 2: Control Information<br />Pt. 3: Sensory Information<br />Pt. 4: Virtual World Object Characteristics<br />Pt. 5: Data Formats for Interaction Devices<br />Pt. 6: Common Types and Tools<br />Pt. 7: Conformance and Reference Software<br />http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-V<br />
  4. 4. Concept of MPEG-V Sensory Information<br />Consumption of multimedia content may stimulate also other senses<br />Vision or audition<br />Olfaction, mechanoreception, equilibrioception, thermoception, …<br />Annotation with metadata providing so-called sensory effects that steer appropriate devices capable of rendering these effects<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />4<br />… giving her/him the sensation of being part of the particular media<br />➪ worthwhile, informative user experience <br />
  5. 5. Sensory Effect Description Language (SEDL)<br />XML Schema-based language for describing sensory effects<br />Basic building blocks to describe, e.g., light, wind, fog, vibration, scent<br />MPEG-V Part 3, Sensory Information<br />Adopted MPEG-21 DIA tools for adding time information (synchronization)<br />Actual effects are not part of SEDL but defined within the Sensory Effect Vocabulary (SEV)<br />Extensibility: additional effects can be added easily w/o affecting SEDL<br />Flexibility: each application domain may define its own sensory effects<br />Description conforming to SEDL :== Sensory Effect Metadata (SEM)<br />May be associated to any kind of multimedia content (e.g., movies, music, Web sites, games)<br />Steer sensory devices like fans, vibration chairs, lamps, etc. via an appropriate mediation device<br />➪ Increase the experience of the user<br />➪ Worthwhile, informative user experience<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />5<br />
  6. 6. Sensory Effect Description Language (cont’d)<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />6<br />SEM ::=[DescriptionMetadata](Declarations|GroupOfEffects|Effect|ReferenceEffect)+<br />Declarations ::= (GroupOfEffects|Effect|Parameter)+<br />GroupOfEffects ::= timestamp EffectDefinitionEffectDefinition (EffectDefinition)*<br />Effect ::= timestamp EffectDefinition<br />EffectDefinition ::= [activate][duration][fade][alt] [priority][intensity][position]<br />[adaptability]<br />
  7. 7. SEVino, SESim, SEMP, and amBX<br />amBX (Ambient Experience) system + SDK<br />Two fan devices, a wrist rumbler, two sound speakers, a subwoofer, two lights, and a wall washer<br />Everything controlled by SEM descriptionsexcept light effect<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />7<br />Annotation Tool: SEVino<br />Simulator: SESim<br />Player: SEMP<br />
  8. 8. Are Sensory Effects Ready for the World Wide Web?<br />How to embed SEM into Web documents?<br />How to interpret/parse SEM within a browser?<br />Directly via JavaScript<br />Browser plug-in <br />How to synchronize with audio/video content of the Web document?<br />HTML5’s video and audio elements and corresponding DOM attributes<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />8<br /><link rel="alternate" type="text/xml”<br />href="http://server.org/sem.xml”media="light, wind, vibration"/><br />
  9. 9. Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />Synchronization with audio/video (cont’d)<br />currentTimeDOM attribute: current playback position in seconds<br />timeupdateevent: indicates that the current playback position changed as part of normal playback <br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />9<br />
  10. 10. Are Sensory Effects Ready for the World Wide Web? (cont’d)<br />How to extract color information automatically?<br />Used for controlling ambient lights<br />currentFramedoes neither exist nor is foreseen in HTML5<br />canvas element allows for manipulating the display’s content<br />(+) Pro: extract the pixel information from the display<br />(-) Con: needs to be added explicitly during authoring<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />10<br />
  11. 11. Conclusions<br />Introduction to MPEG-V Sensory Effects<br />Are Sensory Effects Ready for the World Wide Web? – Yes, they are!<br />Future work<br />Implementation is ongoing<br />Subjective quality assessments<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />11<br />
  12. 12. Thank you for your attention<br />... questions, comments, etc. are welcome …<br />Ass.-Prof. Dipl.-Ing. Dr. Christian Timmerer<br />Klagenfurt University, Department of Information Technology (ITEC)<br />Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA<br />christian.timmerer@itec.uni-klu.ac.at<br />http://research.timmerer.com/<br />Tel: +43/463/2700 3621 Fax: +43/463/2700 3699<br />© Copyright: Christian Timmerer<br />12<br />2010/05/20<br />Christian Timmerer, Klagenfurt University, Austria<br />

×