Serving Multimedia in Web


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Serving Multimedia in Web

  1. 1. Multimedia Concepts
  2. 2. Topics <ul><li>Multimedia Basic Terms </li></ul><ul><li>Multimedia Compression/Decompression </li></ul><ul><li>Multimedia Protocols </li></ul>
  3. 3. Basic Terms <ul><li>Multimedia Definition </li></ul><ul><ul><li>The term multimedia usually implies that at least one of text ( structured/unstructured, hypertext, etc ), graphics (drawings), or image ( discrete media ) is associated with either audio or motion video information ( continuous media ). </li></ul></ul>
  4. 4. Basic Terms <ul><li>Multimedia streaming </li></ul><ul><ul><li>Multimedia streaming is the overlapping the playout of the data at the receiver with the transmission by the sender. </li></ul></ul><ul><ul><ul><li>A video stream consists of a sequence of images or frames. </li></ul></ul></ul><ul><ul><ul><ul><li>A frame consists of a grid of pixels. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>( Table 1 ) </li></ul></ul></ul></ul><ul><ul><ul><li>An audio stream consists of a sequence of audio samples. </li></ul></ul></ul>
  5. 5. Basic Terms Table 1. Hierarchy of multimedia content. Picture element Two-dimensional grid of pixels Sequence of frames over time Synchronized set of streams Set of multimedia sessions Pixel Frame Stream Session Presentation Definition Term
  6. 6. Basic Terms <ul><li>The advantage of streaming is that it can enable easier access to multimedia resources. </li></ul><ul><li>Another possibility is the integration of video and audio with other web-based applications, such as chat and other real-time collaboration tools. </li></ul>
  7. 7. Basic Terms <ul><li>Streaming vs. downloading </li></ul><ul><li>What Is The Difference Between Downloading and Streaming ? </li></ul><ul><ul><li>When you download a video, you have to copy the </li></ul></ul><ul><ul><li>entire file to your hard disk before you can play it. </li></ul></ul>
  8. 8. Basic Terms <ul><ul><li>When the video is streamed , there is a small wait as the stream 'buffers' but there is no need to save the file. </li></ul></ul><ul><ul><li>Streaming is the act of sending media files (audio and/or video) over the Internet from one computer to another computer so that the media plays as it is being delivered. </li></ul></ul>
  9. 9. Basic Terms Figure 2. To hear or view a media file without downloading it <ul><li>Multimedia Streaming </li></ul>Media Encoding Audio Video Animation Clients Send Request To Servers Web Server Send Request to Media Server Media Server Proprietary Format <ul><li>Multicast capable </li></ul><ul><li>More Robust </li></ul><ul><li>Access to Storage </li></ul><ul><li>Relieves Web Server </li></ul>Send Stream To Clients <ul><li>Standalone player </li></ul><ul><li>Java based player </li></ul><ul><li>Browser plug-in player </li></ul><ul><li>Appliance </li></ul><ul><li>Decode </li></ul><ul><li>Buffer </li></ul><ul><li>Sync. </li></ul>
  10. 10. Basic Terms <ul><li>A media stream proceeds through the following stages before it is displayed to a recipient: </li></ul><ul><ul><li>Capturing </li></ul></ul><ul><ul><ul><li>The audio or video stream must be captured from an analog device, such as a microphone or a video camera, and converting to a digital form. </li></ul></ul></ul><ul><ul><ul><li>25 fps (frame per second) for video and 16-bit for audio is suitable. </li></ul></ul></ul>
  11. 11. Basic Terms <ul><ul><li>Encoding </li></ul></ul><ul><ul><ul><li>An encoder converts the raw digital data into a particular audio or video format. </li></ul></ul></ul><ul><ul><li>Storing </li></ul></ul><ul><ul><ul><li>A server may store the encoded stream for future transmission. </li></ul></ul></ul><ul><ul><li>Delivering </li></ul></ul><ul><ul><ul><li>The stream is transmitted to one or more recipients. A live stream may be transmitted as it is captured and encoded, whereas a prerecorded stream is transmitted by a server. </li></ul></ul></ul>
  12. 12. Basic Terms <ul><ul><li>Decoding </li></ul></ul><ul><ul><ul><li>The receiver decodes and displays the data as they arrive. Alternatively, the receiver may store the entire stream before initiating playback. </li></ul></ul></ul><ul><ul><ul><li>( Figure 3 ) </li></ul></ul></ul>
  13. 13. Basic Terms Figure 3. Capturing Video
  14. 14. Basic Terms <ul><li>There are two different types of streaming : </li></ul><ul><ul><li>Progressive download </li></ul></ul><ul><ul><ul><li>The client begins playback of the multimedia file as it is delivered. The file is ultimately stored on the client computer. </li></ul></ul></ul><ul><ul><ul><li>Use standard web server </li></ul></ul></ul><ul><ul><ul><li>Quality is better than real-time streaming </li></ul></ul></ul><ul><ul><li>Real-time streaming </li></ul></ul><ul><ul><ul><li>The multimedia file is delivered to the client computer but the file is not stored on the client computer. </li></ul></ul></ul><ul><ul><ul><li>Require a special streaming server </li></ul></ul></ul>
  15. 15. Basic Terms <ul><li>two different types of real-time streaming : </li></ul><ul><ul><li>Live streaming </li></ul></ul><ul><ul><ul><li>used to deliver a live event while it is occurring. </li></ul></ul></ul><ul><ul><ul><li>Examples: live soccer game, live concerts, live radio, and videoconferences. </li></ul></ul></ul><ul><ul><li>On-demand streaming </li></ul></ul><ul><ul><ul><li>used to deliver archived media streams. </li></ul></ul></ul><ul><ul><ul><li>Examples: video clips, movies, and </li></ul></ul></ul><ul><ul><ul><li>lectures. </li></ul></ul></ul>
  16. 16. Basic Terms <ul><li>Why Streaming Media? </li></ul><ul><ul><li>No waiting for complete downloads. </li></ul></ul><ul><ul><li>Streamed files are not written to disk. </li></ul></ul><ul><ul><li>Presentation of live events is possible. </li></ul></ul>
  17. 17. Basic Terms <ul><li>Major streaming formats </li></ul><ul><ul><li>Microsoft Windows Media </li></ul></ul><ul><ul><li>Real Networks </li></ul></ul><ul><ul><li>Apple Quicktime </li></ul></ul>
  18. 18. Basic Terms Figure 4. Streaming media development process <ul><li>How does streaming work? </li></ul>
  19. 19. Basic Terms Figure 5. Streaming media playback <ul><li>How does streaming work? </li></ul>
  20. 20. Basic Terms Figure 6. Streaming media from a conventional Web server
  21. 21. Basic Terms Figure 7. Realtime Streaming protocol
  22. 22. Basic Terms <ul><li>RTSP States </li></ul><ul><ul><li>SETUP - the server allocates resources for a client session. </li></ul></ul><ul><ul><li>PLAY - the server delivers a stream to a client session. </li></ul></ul><ul><ul><li>PAUSE - the server suspends delivery of a stream. </li></ul></ul><ul><ul><li>TEARDOWN - the server breaks down the connection and releases the resources allocated for the session. </li></ul></ul>
  23. 23. Basic Terms Figure 8. RTSP state machihne <ul><li>RTSP State Machine </li></ul>
  24. 24. Basic Terms RTSP operation
  25. 25. Basic Terms <ul><li>Clip </li></ul><ul><ul><li>Clip is a media file that contains audio, video, or both. </li></ul></ul>
  26. 26. Basic Terms <ul><li>Webcast </li></ul><ul><ul><li>A webcast uses streaming media technology to take a single content source and distribute it to many simultaneous listeners/viewers by broadcasting over the Internet. </li></ul></ul>
  27. 27. Basic Terms <ul><li>Three general methods for delivering content from a server to a client across a network: </li></ul><ul><ul><li>Unicasting </li></ul></ul><ul><ul><ul><li>The server delivers the content to a single client.  </li></ul></ul></ul><ul><ul><li>Broadcasting </li></ul></ul><ul><ul><ul><li>The server delivers the content to all clients, regardless whether they want the content or not.  </li></ul></ul></ul><ul><ul><li>Multicasting </li></ul></ul><ul><ul><ul><li>The server delivers the content to a group of receivers who indicate they wish to receive the content. </li></ul></ul></ul>
  28. 28. Basic Terms <ul><li>Broadcasting </li></ul><ul><ul><li>Broadcast means a piece of information is sent or transmitted from one point to all other points. </li></ul></ul><ul><ul><li>There is just one sender, but the information is simultaneously sent to all connected receivers. </li></ul></ul><ul><ul><li>In telecommunications, broadcasting means propagation of a flow of information from one source to all potential recipients. </li></ul></ul>
  29. 29. Basic Terms <ul><li>Broadcasting </li></ul><ul><ul><li>In networking, a distinction is made between broadcasting and multicasting . </li></ul></ul><ul><ul><ul><li>Broadcasting sends a message to everyone </li></ul></ul></ul><ul><ul><ul><li>on the network. </li></ul></ul></ul><ul><ul><ul><li>Multicasting sends a message to a select list of recipients. </li></ul></ul></ul>
  30. 30. Basic Terms Figure 10. Unicast delivering
  31. 31. Basic Terms Figure 11. Multicast delivering
  32. 32. Basic Terms Figure 12. Unicast/Multicast delivering <ul><li>Unicast/Multicast </li></ul>Host Router Unicast Host Router Multicast
  33. 33. Basic Terms <ul><li>Bit rate </li></ul><ul><ul><li>Bit rate is amount of data that can be carried from one point to another in a given time period (usually a second). </li></ul></ul><ul><ul><li>Bit rate is sometimes called data rate or transfer rate or bandwidth . </li></ul></ul><ul><li>Multiple Bit Rate Encoding </li></ul><ul><ul><li>Combine several streams with different bit rate into a single file </li></ul></ul><ul><ul><li>The appropriate bit rate stream is automatically selected </li></ul></ul><ul><ul><li>( Figure 13 ) </li></ul></ul>
  34. 34. Basic Terms Figure 13. Multiple bit rate encoding
  35. 35. Basic Terms <ul><li>Aspect Ratio </li></ul><ul><ul><li>This is the ratio of width to height that the encoded video will be. </li></ul></ul><ul><ul><li>This information is present in the output video stream and used by the decoder to display the video at the correct aspect ratio. </li></ul></ul><ul><ul><li>The computer display is designed for an aspect ratio of 1.33:1 , which means that the width of the display area is only 1.33 times the height, almost square. </li></ul></ul>
  36. 36. Basic Terms <ul><li>Frame </li></ul><ul><ul><li>Frame means one still picture. </li></ul></ul><ul><ul><li>By changing still pictures (frames) quickly, human eye &quot;thinks&quot; that the video is smooth and can't separate pictures from each others and instead sees smooth video. </li></ul></ul>
  37. 37. Basic Terms <ul><li>Frame rate </li></ul><ul><ul><li>Frame rate is the number of video frames (complete pictures) that will be presented to the viewer each second. </li></ul></ul><ul><ul><li>Human eye can sees smooth video with the frame rate more than ~24 fps (frames per second). </li></ul></ul><ul><ul><li>In American TV system, NTSC, the frame rate is approximately 29.97 fps. </li></ul></ul><ul><ul><li>In European PAL system the frame rate is 25fps. </li></ul></ul>
  38. 38. Basic Terms <ul><li>Frame Buffer </li></ul><ul><ul><li>Frame buffer is a special memory to hold the complete digital representation of the frame to be displayed on a computer screen. </li></ul></ul><ul><ul><li>The frame buffer is scanned line by line by the digital-to-analog converter system of the display. </li></ul></ul>
  39. 39. Basic Terms <ul><li>Color depth </li></ul><ul><ul><li>Color depth or bit depth is the number of bits used to represent the color of a single pixel in a bitmapped image or video frame buffer. </li></ul></ul><ul><ul><li>It is known as bits per pixel (bpp). </li></ul></ul><ul><ul><li>Higher color depth gives a broader range of distinct colors. </li></ul></ul><ul><ul><li>( Table 2 ) </li></ul></ul>
  40. 40. Basic Terms Table 2. Color depth chart. Number of Colors Bit-Depth 16,777,216 (True Color + Alpha Channel) 32 16,777,216 (True Color, SVGA) 24 65,536 (High Color, XGA) 16 256 (VGA) 8 16 (EGA) 4 4 (CGA) 2 2 (monochrome) 1
  41. 41. Basic Terms <ul><li>Jitter </li></ul><ul><ul><li>In transmission technology, jitter refers to the variation of the delay generated by the transmission equipment. </li></ul></ul><ul><ul><li>In data communications, jitter refers to the variation over time of the network transit delay. </li></ul></ul>
  42. 42. Multimedia Compression <ul><li>Lossless compression </li></ul><ul><li>Lossy compression </li></ul><ul><li>Compression/Coding Standards </li></ul><ul><li>Decompression </li></ul><ul><li>Codec </li></ul>
  43. 43. Multimedia Compression <ul><li>Compression is the process of eliminating redundant information to decrease file size. </li></ul><ul><li>Compression converts frames and pixels to mathematical algorithms that the computer can understand. </li></ul><ul><li>Decompression converts mathematical algorithms back to frames and pixels for playback. </li></ul>
  44. 44. Multimedia Compression <ul><li>Two compression methods are: </li></ul><ul><ul><li>Lossless compression </li></ul></ul><ul><ul><ul><li>Run-Length coding </li></ul></ul></ul><ul><ul><ul><li>Huffman coding </li></ul></ul></ul><ul><ul><ul><li>Arithmetic coding </li></ul></ul></ul><ul><ul><li>Lossy compression </li></ul></ul><ul><ul><ul><li>Transform coding </li></ul></ul></ul>
  45. 45. Lossless compression <ul><li>Lossless compression retains all of the data of the original file as it's converted to a smaller file size. </li></ul><ul><li>In lossless compression the information is recovered without any alteration after the decompression stage. </li></ul><ul><li>When a lossless file is opened, algorithms restore all compressed information, creating a duplicate of the source file. </li></ul>
  46. 46. Lossless compression <ul><li>It generally preferred for creating high-quality or professional applications. </li></ul><ul><li>Lossless compression is applied where the accuracy of the information is essential, such as in medical imaging where it's important to retain fine detail. </li></ul><ul><li>Lossless compression is also called bit-preserving compression . </li></ul>
  47. 47. Lossy compression <ul><li>Lossy compression refers to the case where the decompressed information is different from the original uncompressed information. </li></ul><ul><li>With this kind of compression, some of the source file's information is discarded to conserve space. </li></ul>
  48. 48. Lossy compression <ul><li>When the file is decompressed, this information is reconstructed through algorithms. </li></ul><ul><li>This method results in some loss of sound quality or image detail when compared to the original. </li></ul><ul><li>This mode is suitable for most continuous media such as sound and motion video as well as for many images. </li></ul>
  49. 49. Compression/Coding Standards <ul><li>The MPEG Standards </li></ul><ul><ul><li>MPEG standards developed and managed by Motion Picture Experts Group (MPEG) </li></ul></ul><ul><ul><ul><li>MPEG-1: VCD </li></ul></ul></ul><ul><ul><ul><li>MPEG-2: DVD, HDTV </li></ul></ul></ul><ul><ul><ul><li>MPEG-4: Content-based video coding </li></ul></ul></ul><ul><ul><ul><li>MPEG-7: Multimedia indexing and retrieval </li></ul></ul></ul><ul><ul><ul><li>MPEG-21: Multimedia delivery and consumption </li></ul></ul></ul>
  50. 50. Compression/Coding Standards <ul><li>The MPEG-1 </li></ul><ul><ul><li>Released in 1992 </li></ul></ul><ul><ul><li>A standard for coded representation of </li></ul></ul><ul><ul><ul><li>Moving pictures </li></ul></ul></ul><ul><ul><ul><li>Audio </li></ul></ul></ul><ul><ul><ul><li>Combination of above </li></ul></ul></ul><ul><ul><li>Typical application – video CD (VCD) </li></ul></ul>
  51. 51. Compression/Coding Standards <ul><li>The MPEG-2 Standard </li></ul><ul><ul><li>Released in 1994 </li></ul></ul><ul><ul><li>A standard to provide video quality not lower than NTSC/PAL with bit rates target between 2-10 Mbit/s </li></ul></ul><ul><ul><li>Applications </li></ul></ul><ul><ul><ul><li>Digital cable TV distribution </li></ul></ul></ul><ul><ul><ul><li>Networked database service via ATM </li></ul></ul></ul><ul><ul><ul><li>Digital video tape recorder (VTR) </li></ul></ul></ul><ul><ul><ul><li>Satellite and terrestrial digital broadcasting distribution </li></ul></ul></ul><ul><ul><li>It also supports HDTV applications, and so pre-emptied MPEG-3 standard </li></ul></ul>
  52. 52. Compression/Coding Standards <ul><li>The MPEG-4 Standard </li></ul><ul><ul><li>First released in 1998, and targeted for content-based multimedia applications and low bit-rate video coding. </li></ul></ul><ul><ul><li>Algorithms and tools for coding and flexible representation of audio/video to meet the challenges of multimedia applications. </li></ul></ul><ul><ul><li>The objective of low bit-rate video coding was later accomplished by H.264, the convergence of ITU-T H.263 and MPEG-2. </li></ul></ul>
  53. 53. Compression/Coding Standards <ul><li>The MPEG-7 </li></ul><ul><ul><li>First release in 2001 </li></ul></ul><ul><ul><li>Official name: Multimedia Content Description Interface </li></ul></ul><ul><ul><li>Objective: </li></ul></ul><ul><ul><ul><li>To allow efficient search for multimedia content using standardized descriptors </li></ul></ul></ul><ul><ul><li>The main research issues: </li></ul></ul><ul><ul><ul><li>Optimum search engine </li></ul></ul></ul><ul><ul><ul><li>Feature analysis & query design </li></ul></ul></ul>
  54. 54. Compression/Coding Standards <ul><li>The MPEG-21 Standard </li></ul><ul><ul><li>Aim at defining a normative open framework for multimedia delivery and consumption for use by all the players in the delivery and consumption chain. </li></ul></ul>
  55. 55. Decompression <ul><li>Decompression is the process by which compressed information is expanded by addition of the redundant information eliminated at the compression stage. </li></ul><ul><li>After decompression, the resulting information may be identical to the original – lossless compression – or be different – lossy compression . </li></ul>
  56. 56. Codec <ul><li>Codec stands for Coder/Decoder or Compression/Decompression . </li></ul><ul><li>Codec is a piece of software or a driver that is mostly for compression to reduce file size but may also do some formatting. </li></ul><ul><li>Compression is the primary function of the Codec. </li></ul>
  57. 57. Codec <ul><li>With codec, your system recognizes the encoded video/audio format and allows you to play (decode) the audio/video file in a particular format. </li></ul>
  58. 58. Codec Examples <ul><li>Video </li></ul><ul><ul><li>MPEG1, MPEG2, DIVX, WMV(WINDOWS MEDIA VIDEO), MPEG4-H264, RealVideo </li></ul></ul><ul><li>Audio </li></ul><ul><ul><li>MP3,ATRAC, AAC, WMA (WINDOWS MEDIA AUDIO), DTS, RealAudio </li></ul></ul><ul><li>Image </li></ul><ul><ul><li>JPEG, JPEG2000, PNG, GIF </li></ul></ul><ul><li>Data </li></ul><ul><ul><li>ZIP, STUFFIT </li></ul></ul>
  59. 59. Multimedia Protocols <ul><li>TCP (Transmission Control Protocol) </li></ul><ul><li>UDP (User Datagram Protocol) </li></ul><ul><li>RTP (Real-time Transport Protocol) </li></ul><ul><li>RSVP (Resource ReSerVation Protocol) </li></ul><ul><li>( Table 3 ) </li></ul>
  60. 60. Multimedia Protocols MULTIMEDIA CONCEPTS Table 3. Multimedia protocols. Disadvantage Advantage Network Protocol · Complicated request mechanism · Receivers may experience random packet loss for small reservation · Reliable connection · Receiver can obtain different levels of service RSVP · No guarantee for QoS · Header is larger than UDP · More complicated that UDP · No support for congestion control · Support real-time transmission · Provide timing reconstruction, loss detection, security and content identification · Allows retrieval of very interesting network statistics RTP/RTCP · Many network firewalls block UDP data · Need error concealment for video packet loss · No support for congestion control · Cannot be played using popular stream players such as QuickTime · Suitable for streaming · Allows packet drops; if packets arrive late or damaged, streaming will continue · No retransmission needed UDP · Typically need large buffer to handle data rate variation · Loss recovery needs retransmission causing further jitter or skew · No support for multicast · Dominate protocol for data transfer of data over the Internet · Streaming through firewall · Reliable TCP
  61. 61. References <ul><li>B.Krishnamurthy, J. Rexford. “Web Protocols and Practice” , 2001 </li></ul><ul><li>A.Silberschatz, P.Garvin, G.Gange. “ operating system concepts”, 2005 </li></ul><ul><li> </li></ul><ul><li>H.Sun, A Vetro, J. Xin. “ An Overview of Scalable Video Streaming”. 2007 </li></ul><ul><li>Z.Li and M.S.Drew. “Fundamentals of Multimedia”, Prentice-Hall 2004 </li></ul><ul><li>R.Steinmetz, K.Nahrstedt, “Multimedia: Computing, Communications and Applications”, Prentice-Hall 1995 </li></ul>
  62. 62. References <ul><li>T.Vaughan, “Multimedia: Making it Work, 7th Edition”, McGraw-Hill College 2006 </li></ul><ul><li>L.Guan, S.Y.Kung, J.Larsen “Multimedia image and video processing” </li></ul>
  63. 63. References <ul><li>Image Compression </li></ul><ul><ul><li>JPEG: </li></ul></ul><ul><ul><li>JPEG: http:// </li></ul></ul><ul><li>Video Compression </li></ul><ul><ul><li>MPEG-4 pt. 2: </li></ul></ul><ul><ul><li>MPEG-4 pt. 10/H.264/AVC: </li></ul></ul><ul><li>File Formats </li></ul><ul><ul><li>General info: http:// </li></ul></ul><ul><ul><li>Containers: http:// </li></ul></ul><ul><ul><li>MPEG-4 pt. 14: </li></ul></ul><ul><ul><li>Codec list: http:// / </li></ul></ul><ul><li>Audio </li></ul><ul><ul><li>MP3: </li></ul></ul>
  64. 64. References DCT Basis Function Image: http:// , GNU licensed JPEG Example Image: http:// , by Ilmari Karonon at Wikipedia, Creative Commons Attribution-ShareAlike 2.5 License MP3 File Image: