Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. A Study of the Growth and Development of Video Streaming over the Internet Sumitha Bhandarkar Department of Electrical Engineering Texas A&M University College Station, TX Sumitha@cs.tamu.edu ABSTRACT 3, we present a brief discussion about some of the problems associated with using TCP at the transport In this paper we present a technical survey of some of the publica- layer for video streaming applications and some of the tions on streaming video over the Internet. We start with an intro- duction of the challenges and some of the solutions associated simple solutions to overcome these problems. with streaming video over the Internet. We then present briefly the JPEG and MPEG standards for still images and motion pic- 2. RELATED WORK tures, respectively. We follow it up with algorithms for lossless smoothing for compressed video, error-resilient scaleable com- In 5 Wu et al., have presented a comprehensive look pression, congestion control paradigm and unequal protection at the challenges associated with streaming video over codes. The paper is then concluded with a study of the characteri- zation of VBR MPEG streams and a look at whether priority the Internet. In this paper the authors cover six key ar- dropping is beneficial in case of layered video. eas namely – video compression, application-layer QOS control, continuous media distribution services, streaming servers, media synchronization mecha- 1. INTRODUCTION nisms, and protocols for streaming media. In the area of video compression the various scaleable and non- The Internet has come a long way from its humble be- scaleable encoding schemes and their relative merits ginnings as a network of few computers designed to are discussed. The objective of the application-layer allow scientists to share data to the current plethora of QOS control is to provide congestion control and er- interconnected heterogeneous networks. Many of the ror control. In the area of congestion control the au- protocols used in conventional applications like file thors discuss several source-based and receiver-based transfer and web surfing have been studied extensive- rate control and rate shaping schemes. In the area of ly and have reached a mature state. However, applica- error control, FEC, retransmission, error resilient en- tions like streaming video over the Internet are still in coding and error concealment are discussed. The sup- the stage of juvenile development, making this an ex- port from the network in the form of filtering, applica- citing time to study their growth and work on their fu- tion-level multicast and content replication is dis- ture. cussed for continuous media distribution. The require- In this paper we present some of the early work on ments of the streaming servers in terms of a real-time standardizing the image and video compression tech- operating system and special storage systems are niques. We then look at some of the current research highlighted. The need for synchronization at the intra- in the area of providing congestion and error control stream, inter-stream and inter-object levels is dis- for streaming media traffic. We also present briefly an cussed. Finally, the different protocols for streaming analytical model for characterizing the traffic generat- video at the network layer, transport layer and session ed by video streams, which is essentially of variable layer are presented. This is an excellent paper that bit rate. A scheme for smoothing techniques for the provides an overview of the overall architecture, re- variable bit rate traffic is presented. Finally, the ques- quirements and current state of technology for stream- tion of whether priority dropping of packets provides ing video over the Internet. It helps bind the different better performance and incentive characteristics com- components together and lays the groundwork for pared to uniform dropping strategy is pondered. more detailed study of each of the components. The rest of the paper is organized as follows. In sec- The compression scheme for video over the Internet tion 2, we take a look at some of the current work in borrows heavily from the JPEG standard for eliminat- the area of video streaming on the Internet. In section ing spatial redundancy. In 5, G. K. Wallace has pre- sented an overview of the JPEG standard, which we 1
  2. 2. discuss very briefly here. JPEG is the first internation- al standard for compressing gray scale and color con- tinuous-tone still images. The JPEG standard provides both lossy and lossless compression techniques. In the lossy technique, the image to be compressed is divid- ed into 8X8 blocks. Each of these blocks is converted to vector of 64 coefficients by applying forward DCT Encoding and Transmission order of an MPEG transform. These coefficients are then quantized using sequence. (Picture Source: [4]) a quantization table. Even though the JPEG standard provides a default quantization table, the users are al- lowed to use their own optimized tables if needed. Encoding and Transmission order of an MPEG The quantized coefficients are then ordered in a zig- sequence. (Picture Source: [4]) zag fashion and encoded using entropy coding tech- niques like huffman coding or arithmetic coding. In MPEG is a continuously evolving standard and sever- the lossless mode several predictors are provided and al improvements have been added to the initial stan- the difference between the chosen predictor output dard over the past few years. One of the important ad- and the actual pixel value is encoded using one of the ditions is the support for fine-grained scalability entropy coding techniques. The JPEG standard also (FGS), presented in 5. The FGS encoding allows the provides two different techniques namely spectral se- server to stream video to satisfy a wide range of band- lection and successive approximation for progressive width requirements based on just two layers, namely g of the source image. Also, a hierarchical mode of the base layer and the enhancement layer. The base operation allows encoding at different resolutions for layer is encoded using the motion compensation allowing high-resolution images to be accessed by methods described earlier. The enhancement layer on low-resolution display. This type of encoding can also the other hand is encoded using bitplane DCT coding, be used in progressive display of the images. allowing the server the flexibility of transmitting any The MPEG standard presented in 5 takes this com- desired portion of any desired enhancement layer pression one level further for motion pictures by tak- frame. This model particularly lends itself well to lay- ing into account the temporal redundancy between ered multicast scheme where the enhancement layer successive frames. Two types of inter-frame coding can be divided among different multicast channels techniques namely predictive coding and interpolative and the receiver needs to decode only two layers irre- coding which use block-based motion compensation spective of the number of channels it is subscribed to. are utilized to reduce the temporal redundancy, in ad- The interframe coding techniques mentioned above dition to the intra-frame coding which uses DCT results in sequences of frames with different sizes, based compression to reduce spatial redundancy. The and hence causes a lot of fluctuation in the bandwidth intra-frames (I-frames) provide points for random ac- usage by the application using these techniques. A so- cess and are coded to reduce only the spatial redun- lution for reducing the rate fluctuation by lossless dancy. The predictive frames (P-frames) are generat- smoothing of compressed video is presented in 5. The ed as the difference between the actual frame and the aim of this paper is to reduce the rate fluctuation be- frame generated by the motion compensation predic- tween consecutive pictures within a repetitive group tor with reference to an earlier I-frame or another P- of pictures, and not the rate fluctuations due to scene frame. The interpolated frames (B-frames) are pre- changes (smoothing between scenes cannot be loss- dicted based on a past I or P frame as well as interpo- less). In this scheme the size of the picture need not lated from a future I or P frame with motion compen- be known ahead of time. By using a buffer that saves sation. The difference between the original frame and frames from i to i+K-1 before sending the ith frame, the predicted is then encoded for transmission. The and controlling the time and rate at which the ith exact way in which the I, P and B frames are orga- frame is transmitted, this algorithm ensures that the nized is upto the application. An example of one such interframe delay is bounded by a parameter D which sequence is shown in Figure [1]. is fixed a priori. A look-ahead interval H is used to improve the performance of the algorithm. 2
  3. 3. An interesting scheme for streaming multimedia data centive properties of uniform versus priority dropping over multicast channels using Unequal Error Protec- for layered video. The first main conclusion in this tion (UEP) codes is presented in 5. The merits of this paper is that the use of priority dropping does not pro- scheme are not just that using UEP allows error re- vide significantly large improvement in performance covery, but also, it allows for multicast receivers to and hence may not warrant the modification of the join an ongoing multicast session asynchronously. current network infrastructure. Secondly, the incen- Also, the scheme is highly scaleable as the resources tive properties of uniform drop are poor, especially consumed at the server depend on the maximum play- with FIFO queues, and would favor selfish users. out delay allowed, rather than the number of receivers As mentioned before, the use of the inter-frame cod- joining the multicast session. The key to this scheme ing techniques results in variable bit rate traffic for is that the server transmits a UEP encoded stream, MPEG applications. A comprehensive model to cap- rather than the original stream, cyclically until the last ture the behavior of such VBR traffic is presented in receiver has received the last frame. Using UEP, dif- 5. This model captures the characteristics of the dif- ferent original symbols can be retrieved from differ- ferent frames (I, P and B) in MPEG streams by repre- ent number of symbols from a continuous UEP en- senting them as three different random processes. Us- coded stream. ing rigorous mathematical analysis, the authors’ show A scheme for providing flow control as well as error that the composite model made of these three random control has been proposed in 5. In this paper the au- processes exhibits long-range dependence (LRD). thors present a scheme where the prioritized multi- resolution output of the MPEG encoder is first con- 3. DISCUSSION verted to a non-prioritized adaptive FEC stream which is robust in the face of uniform dropping by us- In this section we take a brief look at the problems as- ing a technique called MD-FEC. This is then trans- sociated with using TCP at the transport layer for ported over the Internet using an augmented TCP pro- streaming applications and some simple solutions to tocol called LIMD/H, which uses the history of previ- work around the problem. These solutions are not re- ous losses to determine the level of reduction in the sults of rigorous research and are only meant to guide sending rate, when a congestion notification is re- the direction of future research. ceived. This paper also utilizes the measurements First we look at the problems associated with using done by the transport layer (LIMD/H protocol) to TCP for streaming video applications. We identify guide the decisions made by the application layer three main ones – MD-FEC transcoder. 1. TCP algorithm insists on retransmission of lost Most of the research studied in this paper use the packets. This may result in redundancy and MPEG standard for video compression. In 5 though, wasted bandwidth in case of real-time streaming Tan et al., propose an alternate compression scheme applications that normally use FEC techniques for video. In this scheme 3-D subband coding is used to efficiently recover from single losses. Also, if for compressing a video stream to render it more re- a packet is not recovered by either FEC or re- silient to drops, when the packet dropping is uniform. transmission before its playout deadline, the The authors show that not only is the performance of packet is useless at the receiver anyway. this scheme comparable to that of MPEG-1, its com- putation complexity is lesser making it better suited 2. TCP does not lend itself to be used in IP-multi- for real-time encoding and decoding. A simple rate- cast. When multiple receivers request the same based TCP-friendly protocol is used at the transport stream of video, it is a lot more efficient to mul- layer to provide flow control. ticast the data than send it over multiple unicast sessions. TCP cannot be used, as is, in an IP- Use of layered techniques for streaming multimedia multicast environment, because it is ack-clocked traffic has resulted in proponents for a priority-drop- and generating acks in a multicast environment ping scheme of these packets in the network routers can be a phenomenally complex problem. during times of congestion. Bajaj et al., have present- ed a study based on simulations as well as mathemati- 3. TCP performs congestion control based on con- cal analysis in 5 to compare the performance and in- gestion indication from the network resulting in abrupt variations in sending rate. This would re- 3
  4. 4. quire very large amounts of buffering at the re- ly, it can be shown that the additional buffer space re- ceiver. quired at the receivers (in packets) is bounded by τ × rwnd Now we look at some of the possible simple solu- PBM tions. First we consider the case of retransmission. While it is not easy to work around the redundancy is- where rwnd is the receiver advertised transmission sue, it is relatively easy to turn off retransmission rate and PBM is the minimum packet size of a base lay- when deadlines are missed. A simple solution would er packet. The reason behind this is that, the maxi- be for the receiver to send dummy acknowledgment mum rate at which the TCP sender can send date is to a lost packet, if the packet is not recovered within a limited by min(cwnd,rwnd), where cwnd is the con- threshold α before its playout deadline. The threshold gestion window which can never be increased above value should be chosen carefully such that, if the the bandwidth of the bottle neck link and rwnd is the packet has already been retransmitted before the dum- receiver advertised transmission rate. So the upper my ack reaches the sender, it should be usable. bound on sending rate can never exceed rwnd. Next, we consider the case of IP-multicast. A simple 4. CONCLUSION AND FUTURE WORK solution to this problem is to use application level multicast instead of IP-multicast. It has been shown In this paper we have presented some of the standards by recent research that application layer multicast and solutions for streaming video over the Internet, based on an overlay of peer-to-peer computers is an and discussed the issues related to using TCP with efficient substitution for IP-multicast that requires such streaming applications. The intended future support from the network infrastructure. Moreover, in work is to study some of the standardized streaming application layer multicast, the peers can communi- protocols like RTP and RTSP. cate with each other using TCP without any of the problems associated with IP-multicast. 5. REFERENCES Finally, we look at the problems associated with fluc- [1] G.K. Wallace, “The JPEG Still Picture Compression Standard,” tuations in sending rate. Suppose, we find a way of Communications of the ACM, vol. 34, no. 4, April 1991. letting the application know τ units ahead of time [2] D.L. Gall, “MPEG: A video compression standard for multimedia applications,” Communications of the ACM, vol. 34, no. 4, April about an impending reduction in sending rate, then 1991. the application could send base layer packets during [3] S. S. Lam, S. Chow, and D.K.Y. Yau, “A Lossless Smoothing Algo- rithm for Compressed Video,” IEEE/ACM Transactions on Net- the τ time units. Later, when the TCP protocol cuts working, vol. 4, no. 5, October 1996 the sending rate, the application can send enhance- [4] M. Krunz and S.K. Tripathi, “On the Characterization of VBR ment layer packets at the rate allowed by the gradual- MPEG Streams,” ACM SIGMETRICS, 1997. [5] S. Bajaj, L. Breslau, and S. Shenker, “Uniform versus Priority Drop- ly increasing congestion window. The buffers at the ping for Layered Video,” ACM SIGCOMMM, 1998. receivers could store the base layer packets until their [6] W. Tan and A. Zakhor, “Real-Time Internet Video Using Error Re- play out time. silient Scalable Compression and TCP-Friendly Transport Protocol,” IEEE Transactions on Multimedia, vol. 1, no. 2, 1999. In order to implement this scheme, the TCP protocol [7] R. Puri, K.W. Lee, K. Ramchandran, and V. Bharghavan, “An Inte- grated Source Transcoding and Congestion Control Paradigm for should be able to inform the application τ units of Video Streaming in the Internet,” IEEE Transactions on Multimedia, time before it reduces its congestion window. Since vol. 3, no. 1, March 2001 [8] L. Xu, “Efficient and Scalable On-Demand Data Streaming Using there is no way of know when a packet drop will oc- UEP Codes,” ACM Multimedia, 2001. cur, the only way of implementing this is by delaying [9] H. Radha, M. van der Schaar, and Y. Chen, “The MPEG-4 Fine- Grained Scalable Video Coding Method for Multimedia Streaming the congestion response by τ time units. It has been over IP,” IEEE Transactions on Multimedia, vol. 3, no. 1, March shown in 5 that Delayed congestion Response Proto- 2001. cols are TCP friendly when the delays are small. [10] D. Wu, Y.T. Hou, W. Zhu, Y.Q. Zhang, and J.M. Peha, “Streaming Video over the Internet: Approaches and Directions,” IEEE Transac- In order for this scheme to be effective the buffers at tions on Circuits and Systems for Video Technology, vol. 11, no. 3, March 2001. the receivers should be inflated to hold the base layer [11] S. Bhandarkar, “Delayed Congestion Response Protocols”, Master’s packets transmitted during the τ time units. Informal- Thesis, Texas A&M University, August 2001. 4