Uploaded on


More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. M. Neri, A. Campi , R. Suffritti, F. Grimaccia, P. Sinogas, O. Guye, C. Papin, T. Michalareas, L. Gazdag, I, Rakkolainen SKYMEDIA – UAV-BASED CAPTURING OF HD/3D CONTENT WITH WSN AUGMENTATION FOR IMMERSIVE MEDIA EXPERIENCES
  • 2. Skymedia goals
    • Aims at exploring, designing, and demonstrating a novel multimedia end-to-end architecture focused on immersive media experiences to audience during live events
    • Multimedia Service Platform
      • real-time HD/3D processing
      • augmented HD/3D video
    • Unmanned Aerial Vehicle (UAV) and Wireless Sensors Network (WSN) are enabling technologies
      • UAVs capture HD and 3D videos and images from the sky
      • Wireless sensors are able to acquires side-information be fused with the main audiovisual contents
  • 3. Key technologies
    • Innovative 3D coding methods with MVD (Multi-view plus Depth), to be executed in real-time
    • Advanced transcoding methods that work in real-time to filter and adapt multimedia content to different distribution networks
    • Mid-air screens materializing high resolution screens
      • By means of vaporized water
    • UAV to capture HD video and images from the sky
    • Wireless Sensors to form an Augmentation Network (WSAN) deployed on ground to enrich the multimedia content with side-information
      • To be fused with the main audiovisual content
  • 4. Conceptual Architecture
  • 5. Level Sub-System Architecture
  • 6. UAV-Based Content Capturing
    • Technical challenges
      • an unobtrusive and silent video/image capturing
      • data from a ground sensor network need to be collected and aggregated
    • Progress beyond the state-of-the-art
      • carries one or more cameras
        • Vibrations, 3D/HD real time video images (H.264/AVC)
      • kinds of sensor devices
        • , payload size, limited resources, power consumption
  • 7. Wireless Sensor Augmentation Networks and Data Fusion
    • Technical challenges
      • different types of sensors in dynamic environment with different requirements
        • sport games like soccer, rugby, athletics, or motor sports, etc.
      • aggregate data from both ground sensors and airborne sensors
        • With user generated content
    • Progress beyond the state-of-the-art
      • wireless multi-sensor network
        • multi hop network with either a single or a multi sink structure
        • energy management algorithms
        • improved network protocols
      • data fusion
        • development of tools capable of quickly managing the various datasets and of fusing data
  • 8. Multimedia Service Platform
    • Technical challenges
      • Provide real–time immersive services to end-users platforms
        • different constraints, heterogeneous input/output computing power, storage, screen resolution, user interface, sensors and actuators
    • Progress beyond the state-of-the-art
      • Collects pre-formatted multimedia data
        • view point selection and stereoscopic video enrichment and personalization
        • adaptation according to device (smartphone, HD/3D TV screen)
      • real-time management of the HD/3D (MPEG4 MVC, JPEG2000) processing in relation with WSAN data with user interactivity
        • to choose a point of view
        • metadata that are displayed on the HD/3D screen
    • Video-on-Demand is a mandatory multimedia service
  • 9. Data path for live and VoD services
  • 10. Enabled Services (1/2)
    • Real time HD video content captured from the UAV platfor over the live scene
      • Special real-time transcoding methods is needed to enrich and adapt multimedia content
        • focused on bit-rate reduction, format conversion and resolution optimization
    • Telemetry Data
      • remote measurements regarding live event players parameters
        • delivered to a number of different users
    • Location Based Services
      • Ex. touristic information
  • 11. Enabled Services (2/2)
    • Augmented Reality
      • Live capturing, time constraints are particularly stringent
      • Processing center should be coupled with efficent data gathering network
    • Immersive framework for mobile applications
      • Spatial immersion: utilizing the knowledge of users
      • Psychological immersion: offering the user an experience that is updated and related to events
      • Sensory immersion: utilizing state of the art available sensors on the handset (cameras, GPS, gyroscopes, microphones, tough UI, etc.)
  • 12. Applications and Use Cases
    • Marathon race
      • different kinds of sensors (biosensor)
        • runner’s shoes or suit, and fixed sensors and cameras placed along the marathon path
      • athlete health status, his relative position in the race, shifting from his best previous performance, deviation from the race record and so on
    • Pre-Test
      • Turin Marathon, November 2011
      • “ Tutta dritta”, April 2012
    • Final demonstration
      • Turin Marathon, November 2012
  • 13. Marathon Use Case
  • 14. Applications and Use Cases
    • Outdoor music concert
      • The active part of a concert is a small perimeter event with little movement. The audience of a big concert can, however, cover quite large an area. A freely flying UAV gives more flexibility for filming and other sensors than e.g., cameras on static balloons or on moving wire systems, which can move only very limitedly
    • Motorized sport
      • augment the experience of motorized sports fans
      • avoiding them to become bored during the race
  • 15. Conclusion
    • Multimedia architecture able to create immersive media experiences
    • Key roles
      • UAV platform, enabling content capturing from the sky
      • WSAN to collect sensor data
      • advanced Multimedia Service Platform able to manage in real-time HD/3D processing
    • The project is still ongoing
      • Main test event planned in Turin Mrathon 2012
      • Integration process has been started with preliminary encouraging results
  • 16.
    • Thanks