Multimedia Systems And Applications

10,779 views

Published on

Published in: Business, Technology
2 Comments
4 Likes
Statistics
Notes
No Downloads
Views
Total views
10,779
On SlideShare
0
From Embeds
0
Number of Embeds
69
Actions
Shares
0
Downloads
350
Comments
2
Likes
4
Embeds 0
No embeds

No notes for slide

Multimedia Systems And Applications

  1. 1. ICS 218 -Multimedia Systems and Applications Lecture 1 - Introduction to Multimedia Prof. Nalini Venkatasubramanian [email_address]
  2. 2. Course logistics and details <ul><li>Course Web page - </li></ul><ul><ul><li>http://www.ics.uci.edu/~ics218 </li></ul></ul><ul><li>Lectures - MW 2:00-3:20p.m, ELH 110 </li></ul><ul><li>Course Laboratories - machines on 3rd floor CS labs </li></ul><ul><li>ICS 218 Textbook: </li></ul><ul><ul><ul><li>Multimedia: Computing, Communications and Applications </li></ul></ul></ul><ul><ul><ul><li>Ralf Steinmetz and Klara Nahrstedt, Prentice-Hall Inc. </li></ul></ul></ul><ul><li>Other reading material </li></ul><ul><ul><ul><li>Technical papers and reports </li></ul></ul></ul>
  3. 3. Course logistics and details <ul><li>Homeworks and Assignments </li></ul><ul><ul><li>3 homeworks in the quarter of which 1 or 2 may be programming assignments. </li></ul></ul><ul><li>Tests </li></ul><ul><ul><li>Final Exam - as per UCI course catalog </li></ul></ul><ul><li>Course Project </li></ul><ul><ul><li>Maybe done individually, in groups of 2 or 3(max) </li></ul></ul><ul><ul><li>Potential projects on webpage </li></ul></ul><ul><ul><ul><li>Data representation, multimedia systems and networks, multimedia applications </li></ul></ul></ul>
  4. 4. ICS 218 Grading Policy <ul><li>Homeworks - 30% </li></ul><ul><ul><ul><ul><li>(3 homeworks each worth 10% of the final grade). </li></ul></ul></ul></ul><ul><li>Class Project - 40% of the final grade </li></ul><ul><li>Final exam - 30% of the final grade </li></ul><ul><li>Final assignment of grades will be based on a curve. </li></ul>
  5. 5. Lecture Schedule <ul><ul><li>Weeks 1 and 2: </li></ul></ul><ul><ul><ul><li>Multimedia Representation </li></ul></ul></ul><ul><ul><ul><ul><li>Introduction to Multimedia </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Audio/Image/Video Representation </li></ul></ul></ul></ul><ul><ul><li>Weeks 3 and 4: </li></ul></ul><ul><ul><ul><li>Multimedia Compression </li></ul></ul></ul><ul><ul><ul><ul><li>Encoding and Compression Techniques </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Image Compression (JPEG) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Video Compression (MPEG/MPEG2/MPEG4) </li></ul></ul></ul></ul><ul><ul><li>Week 5 and 6: </li></ul></ul><ul><ul><ul><li>Multimedia Resource Management </li></ul></ul></ul><ul><ul><ul><ul><li>Multimedia Quality of Service and Server Design </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Multimedia Operating Systems - Process Management, Filesystems and buffer management </li></ul></ul></ul></ul>
  6. 6. Course Schedule <ul><ul><li>Weeks 7 and 8: </li></ul></ul><ul><ul><ul><li>Multimedia Communication Systems </li></ul></ul></ul><ul><ul><ul><ul><li>MM Networking </li></ul></ul></ul></ul><ul><ul><ul><ul><li>MM Communication </li></ul></ul></ul></ul><ul><ul><li>Week 9: </li></ul></ul><ul><ul><ul><li>Multimedia Synchronization </li></ul></ul></ul><ul><ul><ul><ul><li>Multimedia Synchronization Concepts </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Multimedia Synchronization Enforcement </li></ul></ul></ul></ul><ul><ul><li>Week 10: </li></ul></ul><ul><ul><ul><li>Multimedia Applications and Services </li></ul></ul></ul><ul><ul><ul><ul><li>Multimedia Video Conferencing </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Multimedia Entertainment Applications </li></ul></ul></ul></ul>
  7. 7. Introduction <ul><li>Multimedia Description </li></ul><ul><li>Why multimedia systems? </li></ul><ul><li>Classification of Media </li></ul><ul><li>Multimedia Systems </li></ul><ul><li>Data Stream Characteristics </li></ul>
  8. 8. Multimedia Description <ul><li>Multimedia </li></ul><ul><ul><ul><li>is an integration of continuous media (e.g. audio, video) and discrete media (e.g. text, graphics, images) through which digital information can be conveyed to the user in an appropriate way. </li></ul></ul></ul><ul><ul><li>Multi </li></ul></ul><ul><ul><ul><li>many, much, multiple </li></ul></ul></ul><ul><ul><li>Medium </li></ul></ul><ul><ul><ul><li>An interleaving substance through which something is transmitted or carried on </li></ul></ul></ul>
  9. 9. Why Multimedia Computing? <ul><ul><li>Application driven </li></ul></ul><ul><ul><ul><li>e.g. medicine, sports, entertainment, education </li></ul></ul></ul><ul><ul><li>Information can often be better represented using audio/video/animation rather than using text, images and graphics alone. </li></ul></ul><ul><ul><li>Information is distributed using computer and telecommunication networks. </li></ul></ul><ul><ul><li>Integration of multiple media places demands on </li></ul></ul><ul><ul><ul><li>computation power </li></ul></ul></ul><ul><ul><ul><li>storage requirements </li></ul></ul></ul><ul><ul><ul><li>networking requirements </li></ul></ul></ul>
  10. 10. Multimedia Information Systems <ul><li>Technical challenges </li></ul><ul><ul><li>Sheer volume of data </li></ul></ul><ul><ul><ul><li>Need to manage huge volumes of data </li></ul></ul></ul><ul><ul><li>Timing requirements </li></ul></ul><ul><ul><ul><li>among components of data computation and communication. </li></ul></ul></ul><ul><ul><ul><li>Must work internally with given timing constraints - real-time performance is required. </li></ul></ul></ul><ul><ul><li>Integration requirements </li></ul></ul><ul><ul><ul><li>need to process traditional media (text, images) as well as continuous media (audio/video). </li></ul></ul></ul><ul><ul><ul><li>Media are not always independent of each other - synchronization among the media may be required. </li></ul></ul></ul>
  11. 11. High Data Volume of Multimedia Information
  12. 12. Technology Incentive <ul><li>Growth in computational capacity </li></ul><ul><ul><ul><li>MM workstations with audio/video processing capability </li></ul></ul></ul><ul><ul><ul><li>Dramatic increase in CPU processing power </li></ul></ul></ul><ul><ul><ul><li>Dedicated compression engines for audio, video etc. </li></ul></ul></ul><ul><li>Rise in storage capacity </li></ul><ul><ul><ul><li>Large capacity disks (several gigabytes) </li></ul></ul></ul><ul><ul><ul><li>Increase in storage bandwidth,e.g. disk array technology </li></ul></ul></ul><ul><li>Surge in available network bandwidth </li></ul><ul><ul><ul><li>high speed fiber optic networks - gigabit networks </li></ul></ul></ul><ul><ul><ul><li>fast packet switching technology </li></ul></ul></ul>
  13. 13. Application Areas <ul><li>Residential Services </li></ul><ul><ul><ul><li>video-on-demand </li></ul></ul></ul><ul><ul><ul><li>video phone/conferencing systems </li></ul></ul></ul><ul><ul><ul><li>multimedia home shopping (MM catalogs, product demos and presentation) </li></ul></ul></ul><ul><ul><ul><li>self-paced education </li></ul></ul></ul><ul><li>Business Services </li></ul><ul><ul><ul><li>Corporate training </li></ul></ul></ul><ul><ul><ul><li>Desktop MM conferencing, MM e-mail </li></ul></ul></ul>
  14. 14. Application Areas <ul><li>Education </li></ul><ul><ul><ul><li>Distance education - MM repository of class videos </li></ul></ul></ul><ul><ul><ul><li>Access to digital MM libraries over high speed networks </li></ul></ul></ul><ul><li>Science and Technology </li></ul><ul><ul><ul><li>computational visualization and prototyping </li></ul></ul></ul><ul><ul><ul><li>astronomy, environmental science </li></ul></ul></ul><ul><li>Medicine </li></ul><ul><ul><ul><li>Diagnosis and treatment - e.g. MM databases that provide support for queries on scanned images, X-rays, assessments, response etc. </li></ul></ul></ul>
  15. 15. Classification of Media <ul><ul><li>Perception Medium </li></ul></ul><ul><ul><ul><li>How do humans perceive information in a computer? </li></ul></ul></ul><ul><ul><ul><ul><li>Through seeing - text, images, video </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Through hearing - music, noise, speech </li></ul></ul></ul></ul><ul><ul><li>Representation Medium </li></ul></ul><ul><ul><ul><li>How is the computer information encoded? </li></ul></ul></ul><ul><ul><ul><ul><li>Using formats for representing and information </li></ul></ul></ul></ul><ul><ul><ul><ul><li>ASCII(text), JPEG(image), MPEG(video) </li></ul></ul></ul></ul><ul><ul><li>Presentation Medium </li></ul></ul><ul><ul><ul><li>Through which medium is information delivered by the computer or introduced into the computer? </li></ul></ul></ul><ul><ul><ul><ul><li>Via I/O tools and devices </li></ul></ul></ul></ul><ul><ul><ul><ul><li>paper, screen, speakers (output media) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>keyboard, mouse, camera, microphone (input media) </li></ul></ul></ul></ul>
  16. 16. Classification of Media (cont.) <ul><ul><li>Storage Medium </li></ul></ul><ul><ul><ul><ul><li>Where will the information be stored? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Storage media - floppy disk, hard disk, tape, CD-ROM etc. </li></ul></ul></ul></ul><ul><ul><li>Transmission Medium </li></ul></ul><ul><ul><ul><ul><li>Over what medium will the information be transmitted? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Using information carriers that enable continuous data transmission - networks </li></ul></ul></ul></ul><ul><ul><ul><ul><li>wire, coaxial cable, fiber optics </li></ul></ul></ul></ul><ul><ul><li>Information Exchange Medium </li></ul></ul><ul><ul><ul><ul><li>Which information carrier will be used for information exchange between different places? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Direct transmission using computer networks </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Combined use of storage and transmission media (e.g. electronic mail). </li></ul></ul></ul></ul>
  17. 17. Media Concepts <ul><li>Each medium defines </li></ul><ul><ul><ul><li>Representation values - determine the information representation of different media </li></ul></ul></ul><ul><ul><ul><ul><li>Continuous representation values (e.g. electro-magnetic waves) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Discrete representation values(e.g. text characters in digital form) </li></ul></ul></ul></ul><ul><ul><ul><li>Representation space determines the surrounding where the media are presented. </li></ul></ul></ul><ul><ul><ul><ul><li>Visual representation space (e.g. paper, screen) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Acoustic representation space (e.g. stereo) </li></ul></ul></ul></ul>
  18. 18. Media Concepts (cont.) <ul><li>Representation dimensions of a representation space are: </li></ul><ul><ul><li>Spatial dimensions: </li></ul></ul><ul><ul><ul><li>two dimensional (2D graphics) </li></ul></ul></ul><ul><ul><ul><li>three dimensional (holography) </li></ul></ul></ul><ul><ul><li>Temporal dimensions: </li></ul></ul><ul><ul><ul><li>Time independent (document) - Discrete media </li></ul></ul></ul><ul><ul><ul><ul><li>Information consists of a sequence of individual elements without a time component. </li></ul></ul></ul></ul><ul><ul><ul><li>Time dependent (movie) - Continuous media </li></ul></ul></ul><ul><ul><ul><ul><li>Information is expressed not only by its individual value but also by its time of occurrence. </li></ul></ul></ul></ul>
  19. 19. Multimedia Systems <ul><li>Qualitative and quantitative evaluation of multimedia systems </li></ul><ul><ul><li>Combination of media </li></ul></ul><ul><ul><ul><li>continuous and discrete. </li></ul></ul></ul><ul><ul><li>Levels of media-independence </li></ul></ul><ul><ul><ul><li>some media types (audio/video) may be tightly coupled, others may not. </li></ul></ul></ul><ul><ul><li>Computer supported integration </li></ul></ul><ul><ul><ul><li>timing, spatial and semantic synchronization </li></ul></ul></ul><ul><ul><li>Communication capability </li></ul></ul>
  20. 20. Data Streams <ul><li>Distributed multimedia communication systems </li></ul><ul><ul><ul><li>data of discrete and continuous media are broken into individual units (packets) and transmitted. </li></ul></ul></ul><ul><li>Data Stream </li></ul><ul><ul><ul><li>sequence of individual packets that are transmitted in a time-dependant fashion. </li></ul></ul></ul><ul><ul><ul><li>Transmission of information carrying different media leads to data streams with varying features </li></ul></ul></ul><ul><ul><ul><ul><li>Asynchronous </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Synchronous </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Isochronous </li></ul></ul></ul></ul>
  21. 21. Data Stream Characteristics <ul><ul><ul><li>Asynchronous transmission mode </li></ul></ul></ul><ul><ul><ul><ul><li>provides for communication with no time restriction </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Packets reach receiver as quickly as possible, e.g. protocols for email transmission </li></ul></ul></ul></ul><ul><ul><ul><li>Synchronous transmission mode </li></ul></ul></ul><ul><ul><ul><ul><li>defines a maximum end-to-end delay for each packet of a data stream. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>May require intermediate storage </li></ul></ul></ul></ul><ul><ul><ul><ul><li>E.g. audio connection established over a network. </li></ul></ul></ul></ul><ul><ul><ul><li>Isochronous transmission mode </li></ul></ul></ul><ul><ul><ul><ul><li>defines a maximum and a minimum end-to-end delay for each packet of a data stream. Delay jitter of individual packets is bounded. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>E.g. transmission of video over a network. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Intermediate storage requirements reduced. </li></ul></ul></ul></ul>
  22. 22. Data Stream Characteristics <ul><ul><li>Data Stream characteristics for continuous media can be based on </li></ul></ul><ul><ul><ul><li>Time intervals between complete transmission of consecutive packets </li></ul></ul></ul><ul><ul><ul><ul><li>Strongly periodic data streams - constant time interval </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Weakly periodic data streams - periodic function with finite period. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Aperiodic data streams </li></ul></ul></ul></ul><ul><ul><ul><li>Data size - amount of consecutive packets </li></ul></ul></ul><ul><ul><ul><ul><li>Strongly regular data streams - constant amount of data </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Weakly regular data streams - varies periodically with time </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Irregular data streams </li></ul></ul></ul></ul><ul><ul><ul><li>Continuity </li></ul></ul></ul><ul><ul><ul><ul><li>Continuous data streams </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Discrete data streams </li></ul></ul></ul></ul>
  23. 23. Classification based on time intervals Strongly periodic data stream Weakly periodic data stream Aperiodic data stream T T T1 T3 T2 T1 T2 T
  24. 24. Classification based on packet size T D1 D1 T D1 D2 D3 D1 D2 D3 D1 D2 D3 Dn Strongly regular data stream Weakly regular data stream Irregular data stream t t t
  25. 25. Classification based on continuity Continuous data stream Discrete data stream D D1 D2 D3 D4 D D1 D2 D3 D4
  26. 26. Logical Data Units <ul><ul><li>Continuous media consist of a time-dependent sequence of individual information units called Logical Data Units (LDU). </li></ul></ul><ul><ul><ul><ul><li>a symphony consists of independent sentences </li></ul></ul></ul></ul><ul><ul><ul><ul><li>a sentence consists of notes </li></ul></ul></ul></ul><ul><ul><ul><ul><li>notes are sequences of samples </li></ul></ul></ul></ul><ul><ul><li>Granularity of LDUs </li></ul></ul><ul><ul><ul><ul><li>symphony, sentence, individual notes, grouped samples, individual samples </li></ul></ul></ul></ul><ul><ul><ul><ul><li>film, clip, frame, raster, pixel </li></ul></ul></ul></ul><ul><ul><li>Duration of LDU: </li></ul></ul><ul><ul><ul><ul><li>open LDU - duration not known in advance </li></ul></ul></ul></ul><ul><ul><ul><ul><li>closed LDU - predefined duration </li></ul></ul></ul></ul>
  27. 27. Granularity of Logical Data Units Film Clip Frame Blocks Pixels

×