Serving Multimedia in Web
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Serving Multimedia in Web

  • 1,061 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,061
On Slideshare
1,061
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
64
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Multimedia Concepts
  • 2. Topics
    • Multimedia Basic Terms
    • Multimedia Compression/Decompression
    • Multimedia Protocols
  • 3. Basic Terms
    • Multimedia Definition
      • The term multimedia usually implies that at least one of text ( structured/unstructured, hypertext, etc ), graphics (drawings), or image ( discrete media ) is associated with either audio or motion video information ( continuous media ).
  • 4. Basic Terms
    • Multimedia streaming
      • Multimedia streaming is the overlapping the playout of the data at the receiver with the transmission by the sender.
        • A video stream consists of a sequence of images or frames.
          • A frame consists of a grid of pixels.
          • ( Table 1 )
        • An audio stream consists of a sequence of audio samples.
  • 5. Basic Terms Table 1. Hierarchy of multimedia content. Picture element Two-dimensional grid of pixels Sequence of frames over time Synchronized set of streams Set of multimedia sessions Pixel Frame Stream Session Presentation Definition Term
  • 6. Basic Terms
    • The advantage of streaming is that it can enable easier access to multimedia resources.
    • Another possibility is the integration of video and audio with other web-based applications, such as chat and other real-time collaboration tools.
  • 7. Basic Terms
    • Streaming vs. downloading
    • What Is The Difference Between Downloading and Streaming ?
      • When you download a video, you have to copy the
      • entire file to your hard disk before you can play it.
  • 8. Basic Terms
      • When the video is streamed , there is a small wait as the stream 'buffers' but there is no need to save the file.
      • Streaming is the act of sending media files (audio and/or video) over the Internet from one computer to another computer so that the media plays as it is being delivered.
  • 9. Basic Terms Figure 2. To hear or view a media file without downloading it
    • Multimedia Streaming
    Media Encoding Audio Video Animation Clients Send Request To Servers Web Server Send Request to Media Server Media Server Proprietary Format
    • Multicast capable
    • More Robust
    • Access to Storage
    • Relieves Web Server
    Send Stream To Clients
    • Standalone player
    • Java based player
    • Browser plug-in player
    • Appliance
    • Decode
    • Buffer
    • Sync.
  • 10. Basic Terms
    • A media stream proceeds through the following stages before it is displayed to a recipient:
      • Capturing
        • The audio or video stream must be captured from an analog device, such as a microphone or a video camera, and converting to a digital form.
        • 25 fps (frame per second) for video and 16-bit for audio is suitable.
  • 11. Basic Terms
      • Encoding
        • An encoder converts the raw digital data into a particular audio or video format.
      • Storing
        • A server may store the encoded stream for future transmission.
      • Delivering
        • The stream is transmitted to one or more recipients. A live stream may be transmitted as it is captured and encoded, whereas a prerecorded stream is transmitted by a server.
  • 12. Basic Terms
      • Decoding
        • The receiver decodes and displays the data as they arrive. Alternatively, the receiver may store the entire stream before initiating playback.
        • ( Figure 3 )
  • 13. Basic Terms Figure 3. Capturing Video
  • 14. Basic Terms
    • There are two different types of streaming :
      • Progressive download
        • The client begins playback of the multimedia file as it is delivered. The file is ultimately stored on the client computer.
        • Use standard web server
        • Quality is better than real-time streaming
      • Real-time streaming
        • The multimedia file is delivered to the client computer but the file is not stored on the client computer.
        • Require a special streaming server
  • 15. Basic Terms
    • two different types of real-time streaming :
      • Live streaming
        • used to deliver a live event while it is occurring.
        • Examples: live soccer game, live concerts, live radio, and videoconferences.
      • On-demand streaming
        • used to deliver archived media streams.
        • Examples: video clips, movies, and
        • lectures.
  • 16. Basic Terms
    • Why Streaming Media?
      • No waiting for complete downloads.
      • Streamed files are not written to disk.
      • Presentation of live events is possible.
  • 17. Basic Terms
    • Major streaming formats
      • Microsoft Windows Media
      • Real Networks
      • Apple Quicktime
  • 18. Basic Terms Figure 4. Streaming media development process
    • How does streaming work?
  • 19. Basic Terms Figure 5. Streaming media playback
    • How does streaming work?
  • 20. Basic Terms Figure 6. Streaming media from a conventional Web server
  • 21. Basic Terms Figure 7. Realtime Streaming protocol
  • 22. Basic Terms
    • RTSP States
      • SETUP - the server allocates resources for a client session.
      • PLAY - the server delivers a stream to a client session.
      • PAUSE - the server suspends delivery of a stream.
      • TEARDOWN - the server breaks down the connection and releases the resources allocated for the session.
  • 23. Basic Terms Figure 8. RTSP state machihne
    • RTSP State Machine
  • 24. Basic Terms RTSP operation
  • 25. Basic Terms
    • Clip
      • Clip is a media file that contains audio, video, or both.
  • 26. Basic Terms
    • Webcast
      • A webcast uses streaming media technology to take a single content source and distribute it to many simultaneous listeners/viewers by broadcasting over the Internet.
  • 27. Basic Terms
    • Three general methods for delivering content from a server to a client across a network:
      • Unicasting
        • The server delivers the content to a single client. 
      • Broadcasting
        • The server delivers the content to all clients, regardless whether they want the content or not. 
      • Multicasting
        • The server delivers the content to a group of receivers who indicate they wish to receive the content.
  • 28. Basic Terms
    • Broadcasting
      • Broadcast means a piece of information is sent or transmitted from one point to all other points.
      • There is just one sender, but the information is simultaneously sent to all connected receivers.
      • In telecommunications, broadcasting means propagation of a flow of information from one source to all potential recipients.
  • 29. Basic Terms
    • Broadcasting
      • In networking, a distinction is made between broadcasting and multicasting .
        • Broadcasting sends a message to everyone
        • on the network.
        • Multicasting sends a message to a select list of recipients.
  • 30. Basic Terms Figure 10. Unicast delivering
  • 31. Basic Terms Figure 11. Multicast delivering
  • 32. Basic Terms Figure 12. Unicast/Multicast delivering
    • Unicast/Multicast
    Host Router Unicast Host Router Multicast
  • 33. Basic Terms
    • Bit rate
      • Bit rate is amount of data that can be carried from one point to another in a given time period (usually a second).
      • Bit rate is sometimes called data rate or transfer rate or bandwidth .
    • Multiple Bit Rate Encoding
      • Combine several streams with different bit rate into a single file
      • The appropriate bit rate stream is automatically selected
      • ( Figure 13 )
  • 34. Basic Terms Figure 13. Multiple bit rate encoding
  • 35. Basic Terms
    • Aspect Ratio
      • This is the ratio of width to height that the encoded video will be.
      • This information is present in the output video stream and used by the decoder to display the video at the correct aspect ratio.
      • The computer display is designed for an aspect ratio of 1.33:1 , which means that the width of the display area is only 1.33 times the height, almost square.
  • 36. Basic Terms
    • Frame
      • Frame means one still picture.
      • By changing still pictures (frames) quickly, human eye "thinks" that the video is smooth and can't separate pictures from each others and instead sees smooth video.
  • 37. Basic Terms
    • Frame rate
      • Frame rate is the number of video frames (complete pictures) that will be presented to the viewer each second.
      • Human eye can sees smooth video with the frame rate more than ~24 fps (frames per second).
      • In American TV system, NTSC, the frame rate is approximately 29.97 fps.
      • In European PAL system the frame rate is 25fps.
  • 38. Basic Terms
    • Frame Buffer
      • Frame buffer is a special memory to hold the complete digital representation of the frame to be displayed on a computer screen.
      • The frame buffer is scanned line by line by the digital-to-analog converter system of the display.
  • 39. Basic Terms
    • Color depth
      • Color depth or bit depth is the number of bits used to represent the color of a single pixel in a bitmapped image or video frame buffer.
      • It is known as bits per pixel (bpp).
      • Higher color depth gives a broader range of distinct colors.
      • ( Table 2 )
  • 40. Basic Terms Table 2. Color depth chart. Number of Colors Bit-Depth 16,777,216 (True Color + Alpha Channel) 32 16,777,216 (True Color, SVGA) 24 65,536 (High Color, XGA) 16 256 (VGA) 8 16 (EGA) 4 4 (CGA) 2 2 (monochrome) 1
  • 41. Basic Terms
    • Jitter
      • In transmission technology, jitter refers to the variation of the delay generated by the transmission equipment.
      • In data communications, jitter refers to the variation over time of the network transit delay.
  • 42. Multimedia Compression
    • Lossless compression
    • Lossy compression
    • Compression/Coding Standards
    • Decompression
    • Codec
  • 43. Multimedia Compression
    • Compression is the process of eliminating redundant information to decrease file size.
    • Compression converts frames and pixels to mathematical algorithms that the computer can understand.
    • Decompression converts mathematical algorithms back to frames and pixels for playback.
  • 44. Multimedia Compression
    • Two compression methods are:
      • Lossless compression
        • Run-Length coding
        • Huffman coding
        • Arithmetic coding
      • Lossy compression
        • Transform coding
  • 45. Lossless compression
    • Lossless compression retains all of the data of the original file as it's converted to a smaller file size.
    • In lossless compression the information is recovered without any alteration after the decompression stage.
    • When a lossless file is opened, algorithms restore all compressed information, creating a duplicate of the source file.
  • 46. Lossless compression
    • It generally preferred for creating high-quality or professional applications.
    • Lossless compression is applied where the accuracy of the information is essential, such as in medical imaging where it's important to retain fine detail.
    • Lossless compression is also called bit-preserving compression .
  • 47. Lossy compression
    • Lossy compression refers to the case where the decompressed information is different from the original uncompressed information.
    • With this kind of compression, some of the source file's information is discarded to conserve space.
  • 48. Lossy compression
    • When the file is decompressed, this information is reconstructed through algorithms.
    • This method results in some loss of sound quality or image detail when compared to the original.
    • This mode is suitable for most continuous media such as sound and motion video as well as for many images.
  • 49. Compression/Coding Standards
    • The MPEG Standards
      • MPEG standards developed and managed by Motion Picture Experts Group (MPEG)
        • MPEG-1: VCD
        • MPEG-2: DVD, HDTV
        • MPEG-4: Content-based video coding
        • MPEG-7: Multimedia indexing and retrieval
        • MPEG-21: Multimedia delivery and consumption
  • 50. Compression/Coding Standards
    • The MPEG-1
      • Released in 1992
      • A standard for coded representation of
        • Moving pictures
        • Audio
        • Combination of above
      • Typical application – video CD (VCD)
  • 51. Compression/Coding Standards
    • The MPEG-2 Standard
      • Released in 1994
      • A standard to provide video quality not lower than NTSC/PAL with bit rates target between 2-10 Mbit/s
      • Applications
        • Digital cable TV distribution
        • Networked database service via ATM
        • Digital video tape recorder (VTR)
        • Satellite and terrestrial digital broadcasting distribution
      • It also supports HDTV applications, and so pre-emptied MPEG-3 standard
  • 52. Compression/Coding Standards
    • The MPEG-4 Standard
      • First released in 1998, and targeted for content-based multimedia applications and low bit-rate video coding.
      • Algorithms and tools for coding and flexible representation of audio/video to meet the challenges of multimedia applications.
      • The objective of low bit-rate video coding was later accomplished by H.264, the convergence of ITU-T H.263 and MPEG-2.
  • 53. Compression/Coding Standards
    • The MPEG-7
      • First release in 2001
      • Official name: Multimedia Content Description Interface
      • Objective:
        • To allow efficient search for multimedia content using standardized descriptors
      • The main research issues:
        • Optimum search engine
        • Feature analysis & query design
  • 54. Compression/Coding Standards
    • The MPEG-21 Standard
      • Aim at defining a normative open framework for multimedia delivery and consumption for use by all the players in the delivery and consumption chain.
  • 55. Decompression
    • Decompression is the process by which compressed information is expanded by addition of the redundant information eliminated at the compression stage.
    • After decompression, the resulting information may be identical to the original – lossless compression – or be different – lossy compression .
  • 56. Codec
    • Codec stands for Coder/Decoder or Compression/Decompression .
    • Codec is a piece of software or a driver that is mostly for compression to reduce file size but may also do some formatting.
    • Compression is the primary function of the Codec.
  • 57. Codec
    • With codec, your system recognizes the encoded video/audio format and allows you to play (decode) the audio/video file in a particular format.
  • 58. Codec Examples
    • Video
      • MPEG1, MPEG2, DIVX, WMV(WINDOWS MEDIA VIDEO), MPEG4-H264, RealVideo
    • Audio
      • MP3,ATRAC, AAC, WMA (WINDOWS MEDIA AUDIO), DTS, RealAudio
    • Image
      • JPEG, JPEG2000, PNG, GIF
    • Data
      • ZIP, STUFFIT
  • 59. Multimedia Protocols
    • TCP (Transmission Control Protocol)
    • UDP (User Datagram Protocol)
    • RTP (Real-time Transport Protocol)
    • RSVP (Resource ReSerVation Protocol)
    • ( Table 3 )
  • 60. Multimedia Protocols MULTIMEDIA CONCEPTS Table 3. Multimedia protocols. Disadvantage Advantage Network Protocol · Complicated request mechanism · Receivers may experience random packet loss for small reservation · Reliable connection · Receiver can obtain different levels of service RSVP · No guarantee for QoS · Header is larger than UDP · More complicated that UDP · No support for congestion control · Support real-time transmission · Provide timing reconstruction, loss detection, security and content identification · Allows retrieval of very interesting network statistics RTP/RTCP · Many network firewalls block UDP data · Need error concealment for video packet loss · No support for congestion control · Cannot be played using popular stream players such as QuickTime · Suitable for streaming · Allows packet drops; if packets arrive late or damaged, streaming will continue · No retransmission needed UDP · Typically need large buffer to handle data rate variation · Loss recovery needs retransmission causing further jitter or skew · No support for multicast · Dominate protocol for data transfer of data over the Internet · Streaming through firewall · Reliable TCP
  • 61. References
    • B.Krishnamurthy, J. Rexford. “Web Protocols and Practice” , 2001
    • A.Silberschatz, P.Garvin, G.Gange. “ operating system concepts”, 2005
    • http://www.micc.unifi.it/delbimbo/documents/prog_prod_multimediale/2
    • H.Sun, A Vetro, J. Xin. “ An Overview of Scalable Video Streaming”. 2007
    • Z.Li and M.S.Drew. “Fundamentals of Multimedia”, Prentice-Hall 2004
    • R.Steinmetz, K.Nahrstedt, “Multimedia: Computing, Communications and Applications”, Prentice-Hall 1995
  • 62. References
    • T.Vaughan, “Multimedia: Making it Work, 7th Edition”, McGraw-Hill College 2006
    • L.Guan, S.Y.Kung, J.Larsen “Multimedia image and video processing”
  • 63. References
    • Image Compression
      • JPEG: http://www.faqs.org/faqs/jpeg-faq/part1/
      • JPEG: http:// en.wikipedia.org/wiki/JPEG
    • Video Compression
      • MPEG-4 pt. 2: http://en.wikipedia.org/wiki/MPEG-4_Part_2
      • MPEG-4 pt. 10/H.264/AVC: http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC
    • File Formats
      • General info: http:// en.wikipedia.org/wiki/File_format
      • Containers: http:// en.wikipedia.org/wiki/Container_format
      • MPEG-4 pt. 14: http://en.wikipedia.org/wiki/.m4a
      • Codec list: http:// www.fourcc.org /
    • Audio
      • MP3: http://en.wikipedia.org/wiki/MP3
  • 64. References DCT Basis Function Image: http:// en.wikipedia.org/wiki/Image:Dctjpeg.png , GNU licensed JPEG Example Image: http:// en.wikipedia.org/wiki/Image:Phalaenopsis_JPEG.jpg , by Ilmari Karonon at Wikipedia, Creative Commons Attribution-ShareAlike 2.5 License MP3 File Image: http://en.wikipedia.org/wiki/Image:Mp3filestructure.jpg