Transport methods in 3DTV--A Survey

Transport Methodsin 3DTV—A SurveyTang kaiApril, 24th, 2011 1

Index• Introduction• 3DTV Broadcast• 3DTV Over IP Networks• Discussion and Conclusion 2

Introduction• Ultimate goal • dynamic holography• Most systems available today • via stereoscopy• Actually, 3DTV systems can be designed to support • fixed-view stereoscopy: only two views • free-view stereoscopy: multiple views 3

Introduction• History of 3-D movie • 1903, first stereoscopic 3-D movie was created • 1922, the first full length stereoscopic movie was shown • in the 1950s, Hollywood started 3-D movie production in big numbers• Consensus: a lasting success • backwards compatible • supports different numbers of users • with affordable 3-D display technologies • requires low additional transport/transmission overhead • perceived quality and viewing comfort is better 4

3DTV Broadcast• Analog Transmission • US • April 29th, 1953: a trial live broadcast of the series SPACE PATROL was run in Los Angeles • viewers with a pair of special polarization lenses • December 19th, 1980: The first “nonexperimental” 3DTV “Miss Sadie Thompson,” and Three Stooges • 3-D Video Corporation developed a system: anaglyph format • April 10th, 1981: musical classic, “Kiss Me Kate.” • 3-D Video Corporation: perfect in color 5

3DTV Broadcast• Analog Transmission • European • 1982: Netherlands two popular-scientific 3-D series • a simple red/green anaglyph format • H.-J. Herbst (Hamburg, Germany) and Philips Research Lab • More than 40 million red/green viewing spectacles were sold • “the TV of the future” was disillusioned • 1983 at the International Audio and Video Fair in Berlin • based on a standard PAL channel chain in two-channel mode • For display, two projectors with orthogonal polarization filters were used • so successful that were continued at IAVF in 1985 and 1987 • Unfortunately, transmission system required custom receiver. 6 LIMITED

3DTV Broadcast• Digital Transmission • Background: ongoing transition from analogue to digital TV services • MPEG developed a new compression technology as part of MPEG-2 • The MPEG-2 multiview profile (MVP) • Left-eye view --- MPEG-2 main profile --- backwards- compatibility • Right-eye enhancement layer using the scalable coding tools • MVP, unfortunately, has not found use in commercially services 7

3DTV Broadcast• Digital Transmission • Some promising attempts • 1998 Nagano Winter Games in Japan • right-eye and left-eye HDTV images @ 45Mbps • projected onto a large screen. Impressing and Powerful • 2002 FIFA World Cup in Korea/Japan • the right-eye and left-eye HDTV images were compressed in side-by-side format using the MPEG-2 Main Profile 8

3DTV Broadcast• Digital Transmission • Fixed-view -> flexible 3-D visual data representation formats • Australian DDD company : “video-plus-depth” representation • combination of monoscopic color video and associated per- pixel depth maps • encodes the depth data low bit rate format • transmitted in the “user data” of an MPEG-2 Transport Stream • receiver : rendered by using depth-image-based rendering (DIBR) 9

3DTV Broadcast• Digital Transmission • European IST project ATTEST • “video-plus-depth” representation • Standard MPEG technologies: H.264/AVC • depth data: 200–300 kbps • overhead for the 3-D visual information is only 10% CMP 2-D 10

3DTV Broadcast• Digital Transmission • European IST project ATTEST • First demonstration based on ATTEST • Diagram as follows 1st demo of a 3DTV service 3-D programs,the “video-plus-depth” 3-D TS Contained two on each contains basedvideo stream • an MPEG-2 coded color data representation formatcoded depth-image DVB-T transmission. • an H.264/AVC using a real sequence. DTV-Recorder-Generator PC with a PCI DVB-T card 11 real-time replay of an offline- Received MPEG-2 TS was demultiplexed in software generated MPEG-2 TS video bit streams were decoded in real-time

3DTV Broadcast• Digital Transmission • “video-plus-depth” representation has been standardized within MPEG as a result of work initiated by Philips and Fraunhofer HHI. • The new standard has been published in two parts: • Specification of the depth format itself is called ISO/IEC 23002-3 (MPEG-C) • a method for transmitting “video-plus-depth” within a conventional MPEG-2 TS has become an amendment (Amd. 2) to ISO/IEC 13818-1 (MPEG-2 Systems). 12

3DTV Over IP Networks• Background • IP is proving to be flexible in accommodating communication services • Classical telephone -> VOIP • Transmission of video over Internet is active in R & D • VoD • 2.5G and 3G offer wireless video service • The IP itself leaves many aspects of the transmission to be defined by other layers of the protocol stack and, • thus, offers flexibility in designing the optimal communications system for various 3-D data 13 representations and encoding schemes.

3DTV Over IP Networks • General Outline • 3DTV streaming architectures• Server Unicasting• Server Multicasting• P2P Unicasting• P2P Multicasting • Protocol • Current state of the art: RTP/UDP/IP 14 • Next generation: RTP/DCCP/IP

3DTV Over IP Networks• Streaming Protocols • Most widely used : RTP over UDP • does not contain any congestion control mechanism • lead to congestion collapse when large volumes of video are delivered • Datagram congestion control protocol (DCCP) is designed as a replacement for UDP for media delivery • TCP minus reliability and in-order packet delivery • UDP plus congestion control, connection setup, and acknowledgements 15

3DTV Over IP Networks• Streaming Protocols • DCCP is a transport protocol that implements bi-directional unicast connections of congestion-controlled, unreliable datagrams. • Despite of the unreliable datagram flow • Reliable handshakes for connection setup/teardown and reliable negotiation of options 16

3DTV Over IP Networks• Streaming Protocols • DCCP also accommodates two congestion control mechanisms. • TCP-like Congestion Control • TCP-Friendly Rate Control(TFRC) • TCP-like Congestion Control • identified by CongestionCCID2 in DCCP • behaves similar to TCP’s AIMD congestion control • halving the congestion window in response to a packet drop • respond quickly to changes in available bandwidth 17 • must tolerate the abrupt changes in the congestion window size

3DTV Over IP Networks• Streaming Protocols • TCP-Friendly Rate Control(TFRC) • identified by CCID3 • a form of equation-based flow control that minimizes abrupt changes in the sending rate while maintaining longer-term fairness with TCP • Appropriate for applications that would prefer a rather smooth sending-rate with a small or moderate receiver buffer • streaming media applications 18

3DTV Over IP Networks• Streaming Protocols • TCP-Friendly Rate Control(TFRC) • Operation: CCID3/TFRC calculates TFRC rate • using the TCP throughput equation • Request gives feedback to sender application • Sender may use this rate information to adjust rate to get better results 19

3DTV Over IP Networks• Streaming Protocols • (exp)RFC for TCP-Friendly Multicast Congestion Control (TFMC) • compute the TFRC rate in a multicast scenario • each receiver computes own TFRC rate as a function of RTT loss rate • server then selects the minimum of these rates • (limited number clients to prevent feedback explosion) • DCCP is the same way doing this. • Hence, future 3DTV over IP services is expected to employ the DCCP protocol with effective video rate adaptation to 20 match the TFRC rate.

3DTV Over IP Networks• Multiview Video Encoding and Rate Allocation/Adaptation • Multiview 3-D video can be represented and encoded • Implicitly: “video-plus-depth” representation (discussed) • Explicitly: in raw form • a trade-off between • random access • ease of rate adaptation • compression efficiency • simulcast coding • scalable simulcast coding • multiview coding • scalable multiview coding 21

3DTV Over IP Networks• Multiview Video Encoding and Rate Allocation/Adaptation • The rate adaptation differs, since rate allocation between views offers new flexibilities. • According to the suppression theory of human visual perception • if the right and left views are transmitted and displayed with unequal spatial, temporal and/or quality resolutions, the overall 3- D video quality is determined by the view with the better resolution • Therefore, rate adaptation may be achieved by • adaptation of the spatial, temporal and/or signal-to-noise (SNR) resolution of one of the views • while encoding/transmitting the other view at full rate. 22

3DTV Over IP Networks• Multiview Video Encoding and Rate Allocation/Adaptation • Several open loop and closed loop rate adaptation strategies • closed loop strategies • client estimates some function of the received signal and feeds it back to the transmitter • The transmitter determines an optimized rate • open loop strategies • transmitter does not use feedback from the receiver 23

3DTV Over IP Networks• Multiview Video Encoding and Rate Allocation/Adaptation • open-loop rate adaptation strategies • First paper: content-adaptive video scaling • Rate adaptation has been achieved by • 1) spatial subsampling; • 2) temporal subsampling; • 3) scaling the quantization step-size; • 4) content-adaptive scaling • content-adaptive video scaling approach • Four categories: high/low temporal spatial detail. • Scaling their resolutions 24 • Experiments show that better compression with better quality

3DTV Over IP Networks• Multiview Video Encoding and Rate Allocation/Adaptation • open-loop rate adaptation strategies • Second paper: adaptive selection of temporal levels and quality layers • video is encoded offline with a predetermined number of spatial, temporal and SNR scalability layers. • Content-aware bit allocation among the views is performed during bit stream extraction by adaptive selection scalability layers • The required bit rate reduction is only applied to one of the views. • Experiments shows that better rate-distortion performance compared to static cases. 25

3DTV Over IP Networks• Multiview Video Encoding and Rate Allocation/Adaptation • closed-loop rate adaptation strategies • rate adaptation is done at the server side by feedback from the user. • First paper: • The user’s head position is tracked and predicted • The system allocates more bandwidth to the selected views in order to render the current viewing angle. • Make use of MVC and SVC • Second paper: • Each view is streamed to a different IP-multicast address • A viewer’s client joins appropriate multicast groups to only receive the 3-D information relevant to its current viewpoint 26

3DTV Over IP Networks• Error Correction and Concealment • Sources: packet losses in the wired or wireless IP links • Wired Internet: Congestion -> packet losses • Wireless Internet: capacity limited by bandwidth of radio spectrum • Noise, interference and fading, error bursts(from mobility) • Joint source and channel coding techniques • Error concealment methods (at the decoder) to limit temporal error propagation 27

3DTV Over IP Networks• Error Correction and Concealment • Common error correction approaches for reliable transmission • ARQ • ACK • Delay, not desirable • FEC • In broadcast and multicast services, channel coding techniques have been widely applied 28

3DTV Over IP Networks• Error Correction and Concealment • First paper: • Macroblock classification into unequally important slice groups • Using FMO tool of H.264/AVC • LT codes are used for error protection for low complexity and advanced performace 29

3DTV Over IP Networks• Error Correction and Concealment • Second paper: • Stereoscopic video streaming using FEC techniques • Frames are classified according to their contribution to overall quality • three layers used for UEP • I-frame • P-frame • Left • Right • To find optimum packetization and UEP strategies • Comparative analysis and simulation of Reed–Solomon (RS) and 30 systematic Luby transform (LT) codes

3DTV Over IP Networks• Error Correction and Concealment • Error concealment algorithm for monoscopic not applicable for stereoscopic. • Based on interpolation -> is not sufficient for not depth info is preserved. • Human perception of errors in 3-D video is different • A small degradation -> significant perceptual distortion • Third paper: an error concealment algorithm • Make full use of characteristic of stereoscopic video • Based on the relativity of prediction mode of right frames -> prediction mode of macroblock • restore the lost macroblock according to the estimated motion 31 vector or disparity vector.

3DTV Over IP Networks• Error Correction and Concealment • capabilities of error concealment • To increase the quality of the reconstructed block • a stereoscopic movie: the two views are highly correlated(why) • information about the corresponding region is highly useful for the reconstruction of the lost block. • corresponding pixel pairs identified using feature matching and principles of epipolar geometry • robust estimation of the transformation parameters is used to 32 educe the negative effect of outliers

3DTV Over IP Networks • 3D Video Streaming Demonstrations • end-to-end prototype system for point-to-point streaming of stereoscopic video over UDP supports the1.over a LAN autostereoscopic Sharp 3-D laptopwith nopacket losses supports a monocular display to demonstrate2.employs the backwardsprotocol stack compatibilityRTP/UDP/IP supports an in-house polarized 3-D projection display system that uses a pair projector 33

Discussion and conclusion• A comprehensive survey of the state-of-the art in transport techniques• While the transport solutions must address backwards compatibility issues with the existing digital TV standards and infrastructure• 3DTV flexible• Current and future research issues for 3-D TV transmission • joint transport and coding • Why • determination of the best rate adaptation method • error resilient video encoding and streaming strategies 34

THEEND 35

http://logicmd.net	921
http://feed.logicmd.net	26
url_unknown	9
http://www.slideshare.net	2
http://cache.baidu.com	1
http://cncc.bingj.com	1
http://webcache.googleusercontent.com	1

Transport methods in 3DTV--A Survey

by Kevin Tong on Jun 01, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

7 Embeds 961

Statistics

Transport methods in 3DTV--A Survey — Presentation Transcript