A multimedia system is characterized by the digital integration of various media types like text, graphics, sound, images and video. It allows for interactivity and uses computer networks for distribution. Ensuring quality of service is important for multimedia over networks due to the time sensitivity and synchronization needs of media. Compression techniques like lossy and lossless are used to reduce multimedia file sizes.

1. Multimedia System Zainal A. Hasibuan

2. Acknowledgment This slide is mainly taken from Multimedia System Tutorial at Hong Kong University by Romeo CK YIP email: [email_address]

3. Covered Topics Fundamentals of Multimedia What is a Multimedia System? Data types in multimedia system Quality of Service (QoS) Lossless and Lossy Compression

4. Multimedia System There is no clear definition of the term "Multimedia System". Multimedia versus Multi Media Common characteristics of a Multimedia System Digital Interactive Networking Integration of text, graphics, sound, image, and video

5. What is Multimedia? media that uses multiple forms of information content and information processing to inform or entertain the (user) audience.[ Wikipedia ] Text Audio Image Animation Video Interactivity

6. What is multimedia? any combination of two or more media, represented in a digital form, sufficiently well integrated to be presented via a single interface, or manipulated by a single computer program. [ Chapman ]

7. What is Multimedia? Key properties of multimedia [ Steinmetz ] : Discrete and continuous media Independent media Computer-controlled systems Integration

8. Discrete and Continuous Media Discrete media composed of time-independent information items. Example: text, graphics, or images. Continuous media consist of time-dependent information items. Example: sound or video.

9. Discrete and Continuous Media Natural Sound Video Still Images Synthetic Sound Animation Text Graphics Continous Discrete Captured (from real world) Synthesized (by computer)

10. Delivery Online Uses a network to send information from one computer to another World Wide Web is the most common form of online delivery Offline Removable storage medium is used to carry data Example: CD-ROM, DVD

11. Software Applications for different media types Images: image editing, painting and drawing Text: editor, layout programs Video: editing and post production Animation: drawing Sound: editing and effects

12. Digital Media Analog Data The signals, which we send each other to communicate, are data. Our daily data have many forms: sound, letters, numbers, and other characters (handwritten or printed), photos, graphics, film. All this data is in its nature analog , which means that it varies in type.

13. Digital Data A digital system is an electronic unit. Therefore, it can only deal with data, which are associated with electricity. That is accomplished using electric switches, which are either off or on. It can be illustrated with regular household switches. If the switch is off, it reads numeral “0”. If it is on, it is read as numeral “1”.

14. Digital Media Bits Each 0 or 1 is called a bit . Bit is an abbreviation of the expression BInary digit, which is derived from the binary number system. Table 1.1 Example of Bits 0 1 bit 1 1 bit 0110 4 bit 01101011 8 bit

15. Digital Media Binary Number System The binary number system is made up of digits. Compare with decimal system (10 digit system), decimal system uses digits 0 through 9, the binary system only uses digits 0 and 1. Table 1.2 Decimal System and Binary System Number in Decimal System Representation in Binary Number System 0 0 2 10 3 11 8 1000

16. Digital Media Units 1 byte = 8 bits 1 kilobytes(kB) = 1024 bytes 1 megabytes(MB) = 1000 kB = 1024000 bytes

17. Digital Media Examples of storage media Hard disk, MO, Compact Disc Pros and Cons of Digital Data Pros: Digital can reproduce, transmit, or copy without introducing loss in quality. Cons: Consume a lot more bandwidth during transmission than analog signal.

18. Data Types in MM system Text Data ASCII stands for American Standard Code for Information Interchange. Computers can only understand numbers, so an ASCII code is the numerical representation of a character.

19. Text Data Figure 2.1 ASCII table and description (copyright: www.asciitable.com)

20. Text Data ASCII uses 7 bits to represent a character. As a result only 127 characters are defined as standard ASCII characters. Characters 128-255 are called extended ASCII characters (Figure 2.2). Figure 2.2 Extended ASCII codes (copyright: www.asciitable.com)

21. Text Data EBCDIC EBCDIC (Extended Binary Coded Decimal Interchange Code) is a character set used on early IBM computers. EBCDIC was first introduced in 1965, it was the new character-coding scheme came with IBM System 360 series. EBCDIC uses 8 bits to represent a character.

22. Text Data UNICODE The Unicode character uses 16 bits to represent a character, thus more than 65000 characters can be represented. While 65000 characters are sufficient for encoding most of the many thousands of characters used in major languages of the world.

23. Sound Data Sound Data A typical compact disc can hold up to 74 minutes of 16 bit, 44.1 kHz audio that is uncompressed – about 650 megabytes .

24. Sound Data Table 2.1 shows how the size of a file is affected by the sampling rate and bit length. The file is a one-minute sound clip, recorded and saved in various forms in the Microsoft Windows WAV file format. Table 2.1 Variation of file size and sampling rate (60 seconds audio clip in MS WAV format) Quality Sampling Rate Resolution File Size CD 44 kHz 16 bit Stereo 10.3 MB 44 kHz 8 bit Stereo 5.18 MB FM Radio 22 kHz 16 bit Stereo 5.18 MB 22 kHz 8 bit Stereo 2.59 MB AM Radio 11 kHz 16 bit Stereo 2.59 MB 11 kHz 8 bit Stereo 1.29 MB

25. Image Data Image Data Images, or pictures, are two-dimensional arrays of data called bitmaps, with each element is called pixel .

26. Image Data Units Dpi - Dots Per Inch Bit Depth - The number of bits used to hold a pixel. Also called color depth and pixel depth , the bit depth determines the number of colors that can be displayed at one time. Table 2.2 Color Depths Color Depth Number of Colors 4 bits 16 8 bits 256 16 bits 65,536 24 bits 16,777,216

27. Video Data Video Data Video, or moving images, is a sequence of images. To create a sense of continuity, video must be played at a rate of at least 25 frames per second (fps).

28. Nature of Digital D ocument T ext (.doc, .txt, .pdf, etc), I mage (.jpg, .png, .gif, .bmp, etc), A udio (.wav, .au, .mp3, etc), V ideo (.avi, .asf, .mov, etc)

29. Quality of Service (QoS) Modern multimedia systems feature networking. Multimedia data over the network consists the following characteristics: Time Critical Synchronization Order of Arrival Tolerance of Error

30. Quality of Service (QoS) QoS can be parameterized from an end-to-end point of view: Bandwidth Required Delay Delay Jitter (Variation of a delay) Reliability (Error rate)

31. Why compression? Keep more stuff in memory (increases speed) Increase data transfer from disk to memory [read compressed data and decompress] is faster than [read uncompressed data] Premise: Decompression algorithms are fast True of the decompression algorithms we use

32. Lossless & Lossy Compression Lossless Compression The original data obtained from lossless compression can be recovered from its compressed data e.g. Huffman Coding Lossy Compression Compressed data of lossy compression cannot be recovered to the original data. e.g. JPEG, MPEG

33. Sample Word Frequency Data (from B. Croft, UMass)

34. Zipf and Term Weighting Luhn (1958) suggested that both extremely common and extremely uncommon words were not very useful for indexing.

35. Summary Common characteristics of multimedia systems: digital, interactive, distributed, and involving a wide range of data types, such as video and audio. QoS as an alternative to “best effort” approach in networking.