M. Neri,  A. Campi , R. Suffritti, F. Grimaccia, P. Sinogas, O. Guye, C. Papin, T. Michalareas, L. Gazdag, I, Rakkolainen ...
Skymedia goals <ul><li>Aims at exploring, designing, and demonstrating a novel multimedia end-to-end architecture focused ...
Key technologies <ul><li>Innovative 3D coding methods with MVD (Multi-view plus Depth), to be executed in real-time </li><...
Conceptual Architecture
Level Sub-System Architecture
UAV-Based Content Capturing <ul><li>Technical challenges </li></ul><ul><ul><li>an unobtrusive and silent video/image captu...
Wireless Sensor Augmentation Networks and Data Fusion <ul><li>Technical challenges </li></ul><ul><ul><li>different types o...
Multimedia Service Platform <ul><li>Technical challenges </li></ul><ul><ul><li>Provide real–time immersive services to end...
Data path for live and VoD services
Enabled Services (1/2) <ul><li>Real time HD video content captured from the UAV platfor over the live scene </li></ul><ul>...
Enabled Services (2/2) <ul><li>Augmented Reality </li></ul><ul><ul><li>Live capturing, time constraints are particularly s...
Applications and Use Cases <ul><li>Marathon race </li></ul><ul><ul><li>different kinds of sensors (biosensor) </li></ul></...
Marathon Use Case
Applications and Use Cases <ul><li>Outdoor music concert </li></ul><ul><ul><li>The active part of a concert is a small per...
Conclusion <ul><li>Multimedia architecture able to create immersive media experiences </li></ul><ul><ul><li>www.ict-skymed...
<ul><li>Thanks </li></ul>
Upcoming SlideShare
Loading in …5
×

Skymedia

1,322 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,322
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Skymedia

  1. 1. M. Neri, A. Campi , R. Suffritti, F. Grimaccia, P. Sinogas, O. Guye, C. Papin, T. Michalareas, L. Gazdag, I, Rakkolainen SKYMEDIA – UAV-BASED CAPTURING OF HD/3D CONTENT WITH WSN AUGMENTATION FOR IMMERSIVE MEDIA EXPERIENCES
  2. 2. Skymedia goals <ul><li>Aims at exploring, designing, and demonstrating a novel multimedia end-to-end architecture focused on immersive media experiences to audience during live events </li></ul><ul><li>Multimedia Service Platform </li></ul><ul><ul><li>real-time HD/3D processing </li></ul></ul><ul><ul><li>augmented HD/3D video </li></ul></ul><ul><li>Unmanned Aerial Vehicle (UAV) and Wireless Sensors Network (WSN) are enabling technologies </li></ul><ul><ul><li>UAVs capture HD and 3D videos and images from the sky </li></ul></ul><ul><ul><li>Wireless sensors are able to acquires side-information be fused with the main audiovisual contents </li></ul></ul>
  3. 3. Key technologies <ul><li>Innovative 3D coding methods with MVD (Multi-view plus Depth), to be executed in real-time </li></ul><ul><li>Advanced transcoding methods that work in real-time to filter and adapt multimedia content to different distribution networks </li></ul><ul><li>Mid-air screens materializing high resolution screens </li></ul><ul><ul><li>By means of vaporized water </li></ul></ul><ul><li>UAV to capture HD video and images from the sky </li></ul><ul><li>Wireless Sensors to form an Augmentation Network (WSAN) deployed on ground to enrich the multimedia content with side-information </li></ul><ul><ul><li>To be fused with the main audiovisual content </li></ul></ul>
  4. 4. Conceptual Architecture
  5. 5. Level Sub-System Architecture
  6. 6. UAV-Based Content Capturing <ul><li>Technical challenges </li></ul><ul><ul><li>an unobtrusive and silent video/image capturing </li></ul></ul><ul><ul><li>data from a ground sensor network need to be collected and aggregated </li></ul></ul><ul><li>Progress beyond the state-of-the-art </li></ul><ul><ul><li>carries one or more cameras </li></ul></ul><ul><ul><ul><li>Vibrations, 3D/HD real time video images (H.264/AVC) </li></ul></ul></ul><ul><ul><li>kinds of sensor devices </li></ul></ul><ul><ul><ul><li>, payload size, limited resources, power consumption </li></ul></ul></ul>
  7. 7. Wireless Sensor Augmentation Networks and Data Fusion <ul><li>Technical challenges </li></ul><ul><ul><li>different types of sensors in dynamic environment with different requirements </li></ul></ul><ul><ul><ul><li>sport games like soccer, rugby, athletics, or motor sports, etc. </li></ul></ul></ul><ul><ul><li>aggregate data from both ground sensors and airborne sensors </li></ul></ul><ul><ul><ul><li>With user generated content </li></ul></ul></ul><ul><li>Progress beyond the state-of-the-art </li></ul><ul><ul><li>wireless multi-sensor network </li></ul></ul><ul><ul><ul><li>multi hop network with either a single or a multi sink structure </li></ul></ul></ul><ul><ul><ul><li>energy management algorithms </li></ul></ul></ul><ul><ul><ul><li>improved network protocols </li></ul></ul></ul><ul><ul><li>data fusion </li></ul></ul><ul><ul><ul><li>development of tools capable of quickly managing the various datasets and of fusing data </li></ul></ul></ul>
  8. 8. Multimedia Service Platform <ul><li>Technical challenges </li></ul><ul><ul><li>Provide real–time immersive services to end-users platforms </li></ul></ul><ul><ul><ul><li>different constraints, heterogeneous input/output computing power, storage, screen resolution, user interface, sensors and actuators </li></ul></ul></ul><ul><li>Progress beyond the state-of-the-art </li></ul><ul><ul><li>Collects pre-formatted multimedia data </li></ul></ul><ul><ul><ul><li>view point selection and stereoscopic video enrichment and personalization </li></ul></ul></ul><ul><ul><ul><li>adaptation according to device (smartphone, HD/3D TV screen) </li></ul></ul></ul><ul><ul><li>real-time management of the HD/3D (MPEG4 MVC, JPEG2000) processing in relation with WSAN data with user interactivity </li></ul></ul><ul><ul><ul><li>to choose a point of view </li></ul></ul></ul><ul><ul><ul><li>metadata that are displayed on the HD/3D screen </li></ul></ul></ul><ul><li>Video-on-Demand is a mandatory multimedia service </li></ul>
  9. 9. Data path for live and VoD services
  10. 10. Enabled Services (1/2) <ul><li>Real time HD video content captured from the UAV platfor over the live scene </li></ul><ul><ul><li>Special real-time transcoding methods is needed to enrich and adapt multimedia content </li></ul></ul><ul><ul><ul><li>focused on bit-rate reduction, format conversion and resolution optimization </li></ul></ul></ul><ul><li>Telemetry Data </li></ul><ul><ul><li>remote measurements regarding live event players parameters </li></ul></ul><ul><ul><ul><li>delivered to a number of different users </li></ul></ul></ul><ul><li>Location Based Services </li></ul><ul><ul><li>Ex. touristic information </li></ul></ul>
  11. 11. Enabled Services (2/2) <ul><li>Augmented Reality </li></ul><ul><ul><li>Live capturing, time constraints are particularly stringent </li></ul></ul><ul><ul><li>Processing center should be coupled with efficent data gathering network </li></ul></ul><ul><li>Immersive framework for mobile applications </li></ul><ul><ul><li>Spatial immersion: utilizing the knowledge of users </li></ul></ul><ul><ul><li>Psychological immersion: offering the user an experience that is updated and related to events </li></ul></ul><ul><ul><li>Sensory immersion: utilizing state of the art available sensors on the handset (cameras, GPS, gyroscopes, microphones, tough UI, etc.) </li></ul></ul>
  12. 12. Applications and Use Cases <ul><li>Marathon race </li></ul><ul><ul><li>different kinds of sensors (biosensor) </li></ul></ul><ul><ul><ul><li>runner’s shoes or suit, and fixed sensors and cameras placed along the marathon path </li></ul></ul></ul><ul><ul><li>athlete health status, his relative position in the race, shifting from his best previous performance, deviation from the race record and so on </li></ul></ul><ul><li>Pre-Test </li></ul><ul><ul><li>Turin Marathon, November 2011 </li></ul></ul><ul><ul><li>“ Tutta dritta”, April 2012 </li></ul></ul><ul><li>Final demonstration </li></ul><ul><ul><li>Turin Marathon, November 2012 </li></ul></ul>
  13. 13. Marathon Use Case
  14. 14. Applications and Use Cases <ul><li>Outdoor music concert </li></ul><ul><ul><li>The active part of a concert is a small perimeter event with little movement. The audience of a big concert can, however, cover quite large an area. A freely flying UAV gives more flexibility for filming and other sensors than e.g., cameras on static balloons or on moving wire systems, which can move only very limitedly </li></ul></ul><ul><li>Motorized sport </li></ul><ul><ul><li>augment the experience of motorized sports fans </li></ul></ul><ul><ul><li>avoiding them to become bored during the race </li></ul></ul>
  15. 15. Conclusion <ul><li>Multimedia architecture able to create immersive media experiences </li></ul><ul><ul><li>www.ict-skymedia.eu </li></ul></ul><ul><li>Key roles </li></ul><ul><ul><li>UAV platform, enabling content capturing from the sky </li></ul></ul><ul><ul><li>WSAN to collect sensor data </li></ul></ul><ul><ul><li>advanced Multimedia Service Platform able to manage in real-time HD/3D processing </li></ul></ul><ul><li>The project is still ongoing </li></ul><ul><ul><li>Main test event planned in Turin Mrathon 2012 </li></ul></ul><ul><ul><li>Integration process has been started with preliminary encouraging results </li></ul></ul>
  16. 16. <ul><li>Thanks </li></ul>

×