This document discusses MPEG-V sensory information and media context and control. It describes four parts of MPEG-V including control information, sensory information, avatar information, and references. Sensory information allows media to stimulate senses like smell, touch, and temperature. It presents a system architecture for sensory information and effect descriptions. The document demonstrates sensory effects like light, sound, and vibration and discusses using them for kinesthetic and tactile experiences.

1. MPEG-VMedia Context and Control N107{82,83,85,86} – Text of ISO/IEC 23006-[1-4] CD Christian Timmerer Klagenfurt University (UNIKLU)  Faculty of Technical Sciences (TEWI) Department of Information Technology (ITEC)  Multimedia Communication (MMC) http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.at

2. MPEG-V System Architecture 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 2 Media context and control Pt. 1: Architecture Pt. 2: Control Information Pt. 3: Sensory Information Pt. 4: Avatar Information

4. light (color, flash), heating, cooling, wind, vibration

5. scent, fog, water sprayer, color correction

7. light (color, flash), heating, cooling, wind, vibration

8. scent, fog, water sprayer, color correction

10. What is MPEG Sensory Information? UMA = Universal Multimedia Access Access to content anywhere, anytime, any device == addressing QoS issues Technical feasible thanks to standards ➙ MPEG-21 DIA UME = Universal Multimedia Experience User-centric approach – takes into account the user Quality of Multimedia Experience Sensorial, e.g., sharpness, brightness Perceptual, e.g., what/where is the content Emotional, e.g., feeling, sensation MPEG-V Sensory Information Consumption of multimedia assets may stimulate also other senses, e.g., olfaction, mechanoreception, or thermoception Achieved by annotating the media resources with sensory effects ➙enhanced, worthwhile, and informative user experience, giving the user the sensation of being part of the actual media 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 5 N9113 N10785

11. Concept of MPEG-V Sensory Information 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 6

12. MPEG-V Sensory InformationSystem Architecture 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 7 Native representation➙ standardized representation Authoring Packaging Scope of Standardization Delivery Standardized representation➙ device capabilities & user preferences Adaptation

13. Committee Draft Structure Sensory Effect Description Language Basic building blocks: GeneralInformation, Declaration, Effect, GroupOfEffects, Parameter, ReferenceEffect Common attributes: XSI (timing), activate, duration, fade, alt, priority, intensity, adaptType, adaptRange Sensory Effect Vocabulary Light (colored, flash), temperature, wind, vibration Water sprayer, scent, fog, shadow, color correction Kinesthetic, tactile 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 8

14. Demo Setup 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 9 Wall-washer / Light Display Light Light Fan Fan Speaker Speaker Vibration Bar Subwoofer

15. Demo Video with Effects 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 10

16. Kinesthetic / Tactile Effects 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 11 ♬♩♪♫ abcdefg Haptic Texture Force Haptic interaction Audio Viewers Television Touch or exploration Manipulation Following movements Dynamics Feeling haptic effects Video Text 3D CG model

17. Kinesthetic / Tactile Effects (cont’d) 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 12 Active interactions Passive interactions

18. MPEG-V SI References N10782, Text of ISO/IEC 23006-1 CD N10783, Text of ISO/IEC 23006-2 CD N10785, Text of ISO/IEC 23006-3 CD N10786, Text of ISO/IEC 23006-4 CD Ad-hoc Group metaverse@lists.uni-klu.ac.at http://lists.uni-klu.ac.at/mailman/listinfo/metaverse Some selected publications J. Cha, I. Oakley, Y.-S. Ho, Y. Kim, J. Ryu, “A Framework of Haptic Broadcasting”, IEEE Multimedia, in print, Sept-Oct. 2009 J. Cha, S.-Y. Kim, Y.-S. Ho, J. Ryu, “3D Video Player System with Haptic Interaction based on Depth Image-Based Representation”, IEEE Transactions on Consumer Electronics, Vol. 52, No.2, pp.477-484, May 2006 M. Waltl, C. Timmerer, H. Hellwagner, A Test-Bed for Quality of Multimedia Experience Evaluation of Sensory Effects, Proceedings of the First International Workshop on Quality of Multimedia Experience (QoMEX 2009), San Diego, USA, July 29-31, 2009. 2009/07/03 Christian Timmerer, Klagenfurt University, Austria 13

19. Thank you for your attention ... questions, comments, etc. are welcome … Ass.-Prof. Dipl.-Ing. Dr. Christian Timmerer Klagenfurt University, Department of Information Technology (ITEC) Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA christian.timmerer@itec.uni-klu.ac.at http://research.timmerer.com/ Tel: +43/463/2700 3621 Fax: +43/463/2700 3699 © Copyright: Christian Timmerer 14 2009/07/03 Christian Timmerer, Klagenfurt University, Austria