1. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
A hybrid architecture for delivery of
panoramic video
Martin Prins, Research Scientist, TNO
O.A. Niamut, R. Van Brandenburg (TNO)
J.F. Macq, N.Verzijp, P. R. Alface (Alcatel-Lucent)
EuroITV – June 2013

2. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Outline
 Some trends and backgrounds
 The Fascinate Delivery Network
 A hybrid approach
 Putting it into practice
 Conclusions

3. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
8K
4K
HD

4. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Trends
8K screening
@ Olympics London ’12
HDTV
Smart TV
1st iPhone 1st iPad
4K TV3DTV

5. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
 Produce format-agnostic audiovisual media and
decide on format at delivery or rendering time
 Allow end-users to interactively view and navigate
around high resolution video panorama with the
accompanying audio automatically changing to match
the selected view
 “Record everything, focus on one perspective”
How do we support both trends?

6. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
7K/2K
Watch on
mobile
Zoom in
Watch in
public

7. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Football
Premier League Game
Chelsea FC – Wolverhampton Wanderers (London, October 2010)
Dance Performance
Carmen Suite
Berlin Philharmonic Orchestra conducted by Sir Simon Rattle &
Dance Compagnie Sasha Waltz (Berlin, May 2012)
Concert
BBC Proms (London, August 2012)
Three Demo Productions (so far)

8. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Network
The FascinatE Consortium

9. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
 Cope with huge bandwidth requirements
 Support interactivity and provide low latency for high QoE
 Content segmentation  tiled video streaming
 In-network rendering  adapt to end-user device
 Support pan/tilt/zoom navigation  virtual cameraman
How do we support both trends in the network?

10. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
The FascinatE Delivery Network
Broadcast & IPTV
Media Internet

11. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Functional components in the delivery network
A/V Proxy:
• interfaces with end-devices
• in-network rendering

12. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
A/V Ingest: the Tiling Approach
ROI streams

13. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Layered
Scene
Layer
Omnicam
Layer
BCam
Scale
HD
Grid
2x2
Grid
4x4
Layer
…
Generic Segment
Time_Segment
Representation
Bitrate 1
Scale
…
Scale
7kx2k
Grid
…
Time_Segment
Time_Segment
Representation
Bitrate 2
Tile
T1
Tile
T2
Tile
T3
Tile
T4
A/V Ingest: Common Tiling Format

14. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Unicast
Multicast
A/V Relay: Publisher/Subscriber
 Rationale: seamlessly combine efficiency of
 Multicast: push content useful for many devices
 Unicast: pull content on-demand for single device
 Fit with next-gen managed video delivery
SegmentedContent
Lowres full
panorama
Hires Tile 1
Hires Tile 2
A/V Relay

15. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Segment
Client
Frame
Combiner
A/V Proxy
2nd Screen for control/display
HTTP server
A/V Relay
HAS delivery of spatial segments
Event Signalling
CDN servers
A/V Relay: Tiled HTTP Adaptive Streaming
 Extension of HAS/DASH with spatial segmentation for tiled streaming
 Control plane signaling for manifest and events
 AV relay implementation in live CDN
Spatial Manifest

16. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
HTTP Adaptive Streaming

17. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
TILED-HAS
MPEG-DASH
Zoom LayerZoom Layer structure
Media Presentation
Period
Period
1920x1080
Period Representation A Segment Info
Adaptation Set
Adaptation Set
Grid 2x2
Segment Info
Representation B
Segment Info
Media Segment
Media Segment
Grid
Layout=2x2
Tile X0Y0
Tile X0Y1
6984x1920
Media Presentation
Layer
OMNICAM
Layer
BROADCAST CAM
... ... ...
Signalling ...
Grid 8x8
...
...
Representation A Segment Info
Segment Info Media Segment
Media Segment
...
Tile
Representation
Representation
...
Media Presentation Description

18. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
A/V Relay: The Hybrid approach
Hybrid A/V
Relay
 Reduce bandwidth usage
 Increase flexibility & scalability
 Integrated solution for delivery of panoramic video

19. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
 Pub/Sub client subscribes to all streams
 Video streams are packaged and time segmented
 Segments are stored and a tiled HAS manifest is created
 The A/V Proxy is signalled of the session
A common tiling format simplifies life

20. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Putting it into practice

21. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
iPad…
Hybrid delivery to a big screen
Putting it into practice
And 2nd Screen

22. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
 Camera ingest (6K x 2K) ~4.5 Gbit/s
 Pub/Sub panorama stream ~ 150 Mbit/s
 85 parallel encodings on a single machine
 Tiled-HAS aggregate 10 – 15 Mbit/s
 Depends on tiling needed to recreate a specific view
 End-to-end delay 15s
 Mostly implementation related
 Interaction delay: instantaneous
 HQ access delay: ~4 seconds
Some figures

23. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Conclusions
 A hybrid approach leverages a common tiling format extending the
flexibility and scalability of delivery of ultra high resolution video.
 By means of content tiling, panoramic video can be delivered to various
types of devices, regardless of screen resolution and bandwidth, whilst
offering full control of navigation by the user.
 HAS based mechanisms like MPEG-DASH can easily be extended to
support ultra high resolution video content such that a user can interact
with it.

24. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Future work

25. A hybrid architecture for delivery of panoramic video
Martin Prins, Research Scientist
EuroITV 2013, June 26th
Questions? More info?
 Martin Prins
 Research Scientist Media Networking
 martin.prins@tno.nl
 +31 (0)88 866 78 16
 Web: http://www.fascinate-project.eu/
 Twitter: Fascinate_Prjct
 Facebook and Vimeo
 Come see our demo’s during the demo session!