Chapter 4: Software That Enables Multimedia <ul><li>Operating systems </li></ul><ul><li>Development software </li></ul><ul...
Operating System Software <ul><li>Device drivers </li></ul><ul><li>Multi-tasking, multi-threading </li></ul><ul><li>Memory...
Device Drivers <ul><li>E.g., Display adapter driver, Firewire card driver, Network card driver, Video Camera Driver, Micro...
Codec’s <ul><li>Compression and Decompression software for video and audio </li></ul><ul><ul><li>“ drivers” for the partic...
Networking - 1 <ul><li>Networking functions are incorporated into operating systems </li></ul><ul><ul><li>Remote disk moun...
Networking - 2 <ul><li>Communication protocols </li></ul><ul><ul><li>Physical layer protocols (ISO layer 1) </li></ul></ul...
Client/Server Architecture - 1 <ul><li>Server </li></ul><ul><ul><li>A process that provides access to particular hardware ...
Client/Server Architecture - 2 <ul><li>E.g., database server </li></ul><ul><ul><li>Computer will have database software </...
Operating System & Multimedia Requirements <ul><li>Development of multimedia </li></ul><ul><li>Running multimedia </li></u...
Multimedia Development Software <ul><li>Software needed to  develop  multimedia </li></ul><ul><ul><li>And software needed ...
Vector vs. Bitmap Graphics - 1 <ul><li>Representation of objects in images </li></ul><ul><ul><li>E.g., a rectangle with a ...
Vector vs. Bitmap Graphics - 2 <ul><li>Space Tradeoff </li></ul><ul><ul><li>Bitmap graphics (uncompressed) take more space...
Video - 1 <ul><li>Video is captured in discrete units called  frames </li></ul><ul><ul><li>Frame: Similar to a still pictu...
Video - 2 <ul><li>Time-dependent medium </li></ul><ul><ul><li>Illusion of motion depends on frame rate </li></ul></ul><ul>...
Video - 3 <ul><li>1024 * 768 = 786, 432 pixels/frame </li></ul><ul><li>786, 432 pixels/frame* 3 bytes/pixel = </li></ul><u...
Animation <ul><li>Similar to video </li></ul><ul><li>Each frame created via software </li></ul><ul><ul><li>not from a came...
Related Techniques <ul><li>Biological animation </li></ul><ul><ul><li>Animal gaits, bipedal motion </li></ul></ul><ul><li>...
MPEG, Quicktime, AVI <ul><li>Standard formats for video and audio information </li></ul><ul><ul><li>Includes techniques/st...
Quicktime, AVI <ul><li>Quicktime </li></ul><ul><ul><li>Primarily Macintosh/Apple </li></ul></ul><ul><ul><li>http://www.qui...
MPEG Standards <ul><li>MPEG-1: </li></ul><ul><ul><li>Parts: Systems, Video, Audio </li></ul></ul><ul><ul><li>Applications:...
MPEG: Video <ul><li>MPEG-1 </li></ul><ul><ul><li>Roughly VHS quality at 1,15 Mbit/s  </li></ul></ul><ul><li>MPEG-2 </li></...
MPEG: Audio <ul><li>MPEG standards have  layers </li></ul><ul><ul><li>Layer: Operating modes with increasing complexity an...
MPEG-2 & MPEG-4: Audio <ul><li>MPEG-2 Advanced Audio Coding (AAC)  </li></ul><ul><ul><li>High-quality audio coding standar...
MPEG-4 <ul><li>Goal: Higher levels of interaction  </li></ul><ul><li>1. represent units of aural, visual or audiovisual co...
MP3 <ul><li>MPEG-1/2 Audio Layer-3 </li></ul><ul><ul><li>MPEG-1 Audio Layer-3 </li></ul></ul><ul><ul><li>MPEG-2 Audio Laye...
Sound Representation <ul><li>On a sample basis </li></ul><ul><ul><li>E.g., about 44 kHz for CD quality audio </li></ul></u...
Example <ul><li>1) With a sampling rate of 44.1 kHz, and 16 bit sound resolution, how many KB does it take to represent 3 ...
Example- Solution (1) <ul><li>44,100 samples/sec * 2 bytes/sample = 88, 200 bytes/sec </li></ul><ul><li>88, 200 bytes/sec ...
Example- Solution (2) <ul><li>Problem is to represent sequences of phonemes </li></ul><ul><ul><li>Each phoneme can take on...
Sound Filtering <ul><li>A sound can be filtered to remove a certain set of frequencies </li></ul><ul><li>Some types of fil...
Java - 1 <ul><li>Sun Microsystems </li></ul><ul><li>Goal: Platform independent programs </li></ul><ul><li>Java code compil...
Java - 2 <ul><li>Efficiency </li></ul><ul><ul><li>Just-in-time compilers </li></ul></ul><ul><ul><ul><li>Compile byte-code ...
Web browsers <ul><li>Process various file types </li></ul><ul><ul><li>HTML </li></ul></ul><ul><ul><li>Media formats: image...
Kappner & Steinmetz (1995): Multimedia platform requirements: <ul><li>Multimedia platform should </li></ul><ul><ul><li>Sup...
Multimedia Delivery Software <ul><li>Two general means of delivery </li></ul><ul><ul><li>Compiled or Interpreted </li></ul...
Upcoming SlideShare
Loading in …5
×

Course notes (.ppt)

2,071 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,071
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
47
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Course notes (.ppt)

  1. 1. Chapter 4: Software That Enables Multimedia <ul><li>Operating systems </li></ul><ul><li>Development software </li></ul><ul><li>Delivery software </li></ul>
  2. 2. Operating System Software <ul><li>Device drivers </li></ul><ul><li>Multi-tasking, multi-threading </li></ul><ul><li>Memory management </li></ul><ul><li>Disk space management </li></ul><ul><li>Processor (CPU) management & scheduling </li></ul><ul><li>Window management </li></ul><ul><li>Servers (e.g., printing, email, web, telnet, ssh) </li></ul><ul><li>User level software </li></ul><ul><ul><li>Includes programs to access and deal with lower levels of operating system (e.g., format a disk, partition a disk, change the priority of a task or process) </li></ul></ul><ul><ul><li>And user level applications to get work done </li></ul></ul><ul><ul><ul><li>E.g., Macromedia Director, Power Point, Web browser </li></ul></ul></ul>
  3. 3. Device Drivers <ul><li>E.g., Display adapter driver, Firewire card driver, Network card driver, Video Camera Driver, Microphone driver </li></ul><ul><li>Devices have specific hardware </li></ul><ul><ul><li>Drivers are specific to the hardware </li></ul></ul><ul><ul><li>Enable the operating system to use that specific hardware </li></ul></ul><ul><li>Device drivers depend </li></ul><ul><ul><li>on operating system </li></ul></ul><ul><ul><li>on the specific device </li></ul></ul><ul><li>Device drivers enable programs </li></ul><ul><ul><li>To be less dependent on hardware; more generic </li></ul></ul>
  4. 4. Codec’s <ul><li>Compression and Decompression software for video and audio </li></ul><ul><ul><li>“ drivers” for the particular format used for the files </li></ul></ul><ul><ul><li>Depend on format, not on hardware </li></ul></ul><ul><li>Within a particular general format (e.g., AVI or Quicktime) </li></ul><ul><ul><li>E.g., with AVI, a Cinepak or Indeo codec can be used </li></ul></ul><ul><ul><li>Sorensen is available with Quicktime </li></ul></ul><ul><ul><li>Many codec’s can be used </li></ul></ul><ul><ul><li>Codec must be on development system </li></ul></ul><ul><ul><li>And on the destination system </li></ul></ul>
  5. 5. Networking - 1 <ul><li>Networking functions are incorporated into operating systems </li></ul><ul><ul><li>Remote disk mounting </li></ul></ul><ul><ul><li>Network device (e.g., printer) access </li></ul></ul><ul><ul><li>Lower levels of network data transmission (e.g., physical, data link layers) </li></ul></ul><ul><li>Multimedia software often depends on networks </li></ul><ul><ul><li>Dynamic loading of codec & other components </li></ul></ul><ul><ul><li>Access to web-based presentations </li></ul></ul>
  6. 6. Networking - 2 <ul><li>Communication protocols </li></ul><ul><ul><li>Physical layer protocols (ISO layer 1) </li></ul></ul><ul><ul><ul><li>E.g., Ethernet, token ring, ATM </li></ul></ul></ul><ul><ul><li>Higher protocol layers </li></ul></ul><ul><ul><ul><li>TCP/IP (layers 2 & 3) </li></ul></ul></ul><ul><ul><ul><li>SMTP – simple mail transfer protocol (layer 6) </li></ul></ul></ul><ul><ul><ul><li>HTTP – hypertext transfer protocol (layer 6) </li></ul></ul></ul><ul><li>Performance objectives </li></ul><ul><ul><li>Accuracy, transmission speed, error correction, user-perceived delays, overall throughput </li></ul></ul>
  7. 7. Client/Server Architecture - 1 <ul><li>Server </li></ul><ul><ul><li>A process that provides access to particular hardware or software resources </li></ul></ul><ul><ul><ul><li>Process: A program that runs independently on a computer </li></ul></ul></ul><ul><li>Often used in the context of networks </li></ul><ul><ul><li>Used within software design generally, as well </li></ul></ul><ul><li>“ Server” may designate a software process </li></ul><ul><ul><li>Or may designate a computer (hardware) </li></ul></ul><ul><ul><li>Depends on the kind of service being provided </li></ul></ul>
  8. 8. Client/Server Architecture - 2 <ul><li>E.g., database server </li></ul><ul><ul><li>Computer will have database software </li></ul></ul><ul><ul><li>May also have additional hardware resources (e.g., more disk space) </li></ul></ul><ul><li>E.g., a software license server (“metering”) </li></ul><ul><ul><li>Enables some maximum number of people to be using a software resource at a given time. </li></ul></ul><ul><li>Server receives requests </li></ul><ul><ul><li>Remote requests from a client process on another computer </li></ul></ul><ul><ul><li>Local requests from a client process on same computer </li></ul></ul><ul><ul><li>Requests are sometimes for data sometimes for programs </li></ul></ul>
  9. 9. Operating System & Multimedia Requirements <ul><li>Development of multimedia </li></ul><ul><li>Running multimedia </li></ul><ul><ul><li>Some operating systems may be more suited for running multimedia </li></ul></ul><ul><ul><ul><li>E.g., BeOS has been touted as a multimedia operating system </li></ul></ul></ul><ul><li>Issues include: </li></ul><ul><ul><li>Memory (RAM) required </li></ul></ul><ul><ul><li>Operating system availability </li></ul></ul><ul><ul><li>Application availability </li></ul></ul><ul><ul><li>Speed </li></ul></ul><ul><ul><li>Networking </li></ul></ul>
  10. 10. Multimedia Development Software <ul><li>Software needed to develop multimedia </li></ul><ul><ul><li>And software needed to deliver </li></ul></ul><ul><ul><li>E.g., Shockwave playback in a browser has different requirements than running Macromedia Director </li></ul></ul><ul><li>Development </li></ul><ul><ul><li>Preproduction, production, postproduction </li></ul></ul><ul><li>Types of development software </li></ul><ul><ul><li>2D image capture, scanning & processing; Digital video capture & processing </li></ul></ul><ul><ul><li>Animation; 3D modeling, rendering; Sound capture & processing; Text; Web development; Authoring </li></ul></ul>
  11. 11. Vector vs. Bitmap Graphics - 1 <ul><li>Representation of objects in images </li></ul><ul><ul><li>E.g., a rectangle with a black outline and a gray interior </li></ul></ul><ul><li>Bitmap (raster) graphics </li></ul><ul><ul><li>Specify value for each pixel </li></ul></ul><ul><ul><li>Application data file (e.g., Photoshop) stores collection of pixels making up rectangle </li></ul></ul><ul><ul><ul><li>E.g., 300 x 600 = 180,000 pixels of rectangle </li></ul></ul></ul><ul><ul><ul><li>Requires 176KB to 527KB of data (uncompressed) </li></ul></ul></ul><ul><li>Vector graphics </li></ul><ul><ul><li>Use rectangle drawing tool & color-fill </li></ul></ul><ul><ul><li>Application program generates a data file that represents rectangle as a command and parameters </li></ul></ul><ul><ul><li>E.g., </li></ul></ul><ul><ul><ul><li>Rectangle, 0, 0, 300, 600, black, gray (32) </li></ul></ul></ul><ul><ul><li>Requires about 100 bytes of information </li></ul></ul>
  12. 12. Vector vs. Bitmap Graphics - 2 <ul><li>Space Tradeoff </li></ul><ul><ul><li>Bitmap graphics (uncompressed) take more space </li></ul></ul><ul><ul><li>Vector graphics take less space </li></ul></ul><ul><li>Speed Tradeoff </li></ul><ul><ul><li>Bitmap graphics (uncompressed) are drawn faster </li></ul></ul><ul><ul><li>Vector graphics need to be decoded and then the bitmap (pixels) generated </li></ul></ul><ul><ul><ul><li>Will be slower than bitmap representation </li></ul></ul></ul><ul><ul><ul><li>An issue on slower computers or if a very large number of objects need to be displayed </li></ul></ul></ul><ul><li>Expressiveness </li></ul><ul><ul><li>Some objects can’t be easily drawn in vector fashion </li></ul></ul><ul><li>Contemporary drawing programs may provide both vector and bitmap graphics </li></ul>
  13. 13. Video - 1 <ul><li>Video is captured in discrete units called frames </li></ul><ul><ul><li>Frame: Similar to a still picture </li></ul></ul><ul><li>Temporal and spatial compression </li></ul><ul><ul><li>Spatial: within a frame </li></ul></ul><ul><ul><li>Temporal: across frames </li></ul></ul><ul><li>Key-frame concept in temporal compression </li></ul><ul><ul><li>Some compression methods use this form of coding </li></ul></ul><ul><ul><li>Store information on one frame at particular intervals </li></ul></ul><ul><ul><ul><li>This is a “key frame” </li></ul></ul></ul><ul><ul><li>Then store changes from this point </li></ul></ul><ul><ul><li>Uses frame differencing </li></ul></ul><ul><ul><li>Makes use of common background often shared between multiple frames </li></ul></ul>
  14. 14. Video - 2 <ul><li>Time-dependent medium </li></ul><ul><ul><li>Illusion of motion depends on frame rate </li></ul></ul><ul><ul><ul><li>Don’t perceive motion below about 15 fps </li></ul></ul></ul><ul><ul><li>Also: animation and sound </li></ul></ul><ul><li>Time-dependent media place high computational demands on a computer </li></ul><ul><ul><li>E.g., with 1024 x 768 image resolution, and 24 bit color depth @ 30 fps, what is the (apparent) data transfer rate in KB/second? </li></ul></ul>
  15. 15. Video - 3 <ul><li>1024 * 768 = 786, 432 pixels/frame </li></ul><ul><li>786, 432 pixels/frame* 3 bytes/pixel = </li></ul><ul><li>2,359,296 bytes/frame </li></ul><ul><li>2,359,296 bytes/frame * 30 frame/sec = </li></ul><ul><li>70,778,880 bytes/sec </li></ul><ul><li>(70,778,880 bytes/sec) / (1024 bytes/KB) = </li></ul><ul><li>69,120 KB/sec </li></ul><ul><li>(Apparent: because this is without any data compression) </li></ul>
  16. 16. Animation <ul><li>Similar to video </li></ul><ul><li>Each frame created via software </li></ul><ul><ul><li>not from a camera source </li></ul></ul><ul><li>History </li></ul><ul><ul><li>traditionally done by hand, frame by frame </li></ul></ul><ul><ul><li>E.g., cel-based, or clay-mation </li></ul></ul><ul><li>Model-based animation </li></ul><ul><ul><li>Online (real-time) versus offline </li></ul></ul><ul><ul><li>Model-based: software representation of world </li></ul></ul><ul><ul><li>Online: Frames are generated on-the-fly, in response to dynamic changes of the model </li></ul></ul><ul><ul><li>Offline: Frames generated in response to model changes </li></ul></ul><ul><ul><ul><li>No requirement for real-time algorithms </li></ul></ul></ul>
  17. 17. Related Techniques <ul><li>Biological animation </li></ul><ul><ul><li>Animal gaits, bipedal motion </li></ul></ul><ul><li>Morphing </li></ul><ul><ul><li>Gradual changes in one image to transform it to a second image </li></ul></ul><ul><ul><li>Applying changes gradually </li></ul></ul><ul><ul><ul><li>E.g., taking a few seconds @ 30 fps </li></ul></ul></ul><ul><ul><ul><li>Can be used for animation purposes </li></ul></ul></ul><ul><li>VRML </li></ul><ul><ul><li>Virtual reality modeling language </li></ul></ul><ul><ul><li>&quot;an open standard for 3D multimedia and shared virtual worlds on the Internet.&quot; </li></ul></ul><ul><ul><ul><ul><li>http://home.hiwaay.net/~crispen/vrmlworks/faq/faq1.html#q1 </li></ul></ul></ul></ul><ul><ul><li>“ Briefly, VRML 1.0 worlds are static. VRML 2.0 worlds can move and interact with the visitor to those worlds.” </li></ul></ul>
  18. 18. MPEG, Quicktime, AVI <ul><li>Standard formats for video and audio information </li></ul><ul><ul><li>Includes techniques/standards for compression, decompression </li></ul></ul><ul><li>MPEG </li></ul><ul><ul><li>http://mpeg.telecomitalialab.com/mpeg_general.htm </li></ul></ul><ul><ul><li>http://mpeg.telecomitalialab.com/ </li></ul></ul><ul><ul><li>MPEG Audio </li></ul></ul><ul><ul><ul><li>For a paper on this, see http://citeseer.nj.nec.com/421409.html </li></ul></ul></ul><ul><ul><li>ISO/IEC standardization </li></ul></ul><ul><ul><li>Open standard </li></ul></ul><ul><ul><ul><li>Specification is available to everyone (for a fee) </li></ul></ul></ul><ul><ul><ul><li>No single company “owns” the standard </li></ul></ul></ul>
  19. 19. Quicktime, AVI <ul><li>Quicktime </li></ul><ul><ul><li>Primarily Macintosh/Apple </li></ul></ul><ul><ul><li>http://www.quicktime.com </li></ul></ul><ul><li>AVI </li></ul><ul><ul><li>Primarily Windows/Microsoft </li></ul></ul><ul><ul><li>AVI: “Audio Video Interleave ” </li></ul></ul><ul><ul><li>E.g., see http://www.jmcgowan.com/avi.html </li></ul></ul>
  20. 20. MPEG Standards <ul><li>MPEG-1: </li></ul><ul><ul><li>Parts: Systems, Video, Audio </li></ul></ul><ul><ul><li>Applications: MP3, Video CD </li></ul></ul><ul><li>MPEG-2 </li></ul><ul><ul><li>Parts: Systems, Video, Audio, Conformance, Reference Software Digital Storage Media Command and Control (DSM-CC), Advanced Audio Coding (AAC), Real Time Interface </li></ul></ul><ul><ul><li>MP3 , Digital Television set top boxes, Digital Versatile Discs (DVD). </li></ul></ul><ul><li>MPEG-3 (See MPEG-2) </li></ul><ul><li>MPEG-4 </li></ul><ul><ul><li>Proposed fixed & mobile web standard </li></ul></ul><ul><li>MPEG-7: Multimedia Content Description </li></ul><ul><li>MPEG-21: Multimedia Framework </li></ul>
  21. 21. MPEG: Video <ul><li>MPEG-1 </li></ul><ul><ul><li>Roughly VHS quality at 1,15 Mbit/s </li></ul></ul><ul><li>MPEG-2 </li></ul><ul><ul><li>“ MPEG-2 Video was at least good, and in many cases even better than standards or specifications developed for high bitrate or studio applications” </li></ul></ul><ul><ul><li>http://mpeg.telecomitalialab.com/standards/mpeg-2/mpeg-2.htm </li></ul></ul>
  22. 22. MPEG: Audio <ul><li>MPEG standards have layers </li></ul><ul><ul><li>Layer: Operating modes with increasing complexity and performance </li></ul></ul><ul><li>MPEG-1 provides </li></ul><ul><ul><li>Mono and stereo coding at 32, 44.1, and 48 kHz sampling rate </li></ul></ul><ul><ul><li>Layer I, II, III: varying bit rates from 32 to 448 kbit/s </li></ul></ul><ul><ul><li>Optimized for 128 kbits/s stero </li></ul></ul><ul><li>MPEG-2 BC provides </li></ul><ul><ul><li>Backwards compatible multichannel extension to MPEG-1 </li></ul></ul><ul><ul><li>Up to 5 channels plus a 'low frequent enhancement' channel </li></ul></ul><ul><ul><li>Bit rate range is extended up to about 1 Mbit/s; </li></ul></ul><ul><ul><li>an extension of MPEG-1 towards lower sampling rates 16, 22.05, and 24 kHz for bitrates from 32 to 256 kbit/s (Layer I) and from 8 to 160 kbit/s (Layer II & Layer III). </li></ul></ul><ul><li>http://mpeg.telecomitalialab.com/faq/audio.htm#5 </li></ul>
  23. 23. MPEG-2 & MPEG-4: Audio <ul><li>MPEG-2 Advanced Audio Coding (AAC) </li></ul><ul><ul><li>High-quality audio coding standard for 1 to 48 channels </li></ul></ul><ul><ul><li>Sampling rates of 8 to 96 kHz </li></ul></ul><ul><ul><li>Multichannel, multilingual, and multiprogram capabilities </li></ul></ul><ul><ul><li>Bit rates from 8 kbit/s to more thanf 160 kbit/s/channel </li></ul></ul><ul><ul><li>Multiple encode/decode cycles </li></ul></ul><ul><ul><li>Three profiles (layers) of AAC provide varying levels of complexity and scalability. </li></ul></ul><ul><li>MPEG-4 will provide </li></ul><ul><ul><li>coding and composition of natural and synthetic audio objects at a very wide range of bit rates. </li></ul></ul><ul><ul><li>Emphasis on new functionalities </li></ul></ul>
  24. 24. MPEG-4 <ul><li>Goal: Higher levels of interaction </li></ul><ul><li>1. represent units of aural, visual or audiovisual content, called &quot;media objects&quot;. These media objects can be of natural or synthetic origin; this means they could be recorded with a camera or microphone, or generated with a computer; </li></ul><ul><li>2. describe the composition of these objects to create compound media objects that form audiovisual scenes; </li></ul><ul><li>3. multiplex and synchronize the data associated with media objects, so that they can be transported over network channels providing a QoS appropriate for the nature of the specific media objects; and </li></ul><ul><li>4. interact with the audiovisual scene generated at the receiver’s end. </li></ul><ul><li>http://mpeg.telecomitalialab.com/standards/mpeg-4/mpeg-4.htm </li></ul>
  25. 25. MP3 <ul><li>MPEG-1/2 Audio Layer-3 </li></ul><ul><ul><li>MPEG-1 Audio Layer-3 </li></ul></ul><ul><ul><li>MPEG-2 Audio Layer-3 (successor) </li></ul></ul><ul><li>MP3 and AAC Explained </li></ul><ul><ul><li>http://citeseer.nj.nec.com/421409.html </li></ul></ul><ul><li>MP3  MPEG-3! </li></ul>
  26. 26. Sound Representation <ul><li>On a sample basis </li></ul><ul><ul><li>E.g., about 44 kHz for CD quality audio </li></ul></ul><ul><ul><li>Each sample is a number </li></ul></ul><ul><ul><ul><li>gives the amplitude of the analog sound wave at that point </li></ul></ul></ul><ul><li>On a feature basis </li></ul><ul><ul><li>E.g., frequency, duration, musical note </li></ul></ul><ul><ul><li>Also possible: for speech, phonemes </li></ul></ul><ul><ul><ul><li>Phoneme is a unit of sound in a particular language </li></ul></ul></ul><ul><ul><ul><li>Phonemes may sound differently dependent on context </li></ul></ul></ul><ul><ul><ul><li>E.g. /p/ ( p ersonal, p aternal, com p uter, ra p id), /b/, and /s/ </li></ul></ul></ul><ul><li>Difference is analogous to the difference between bitmapped (raster) & vector graphics </li></ul><ul><li>Tradeoffs– Similar to those between bitmap & vector graphics </li></ul>
  27. 27. Example <ul><li>1) With a sampling rate of 44.1 kHz, and 16 bit sound resolution, how many KB does it take to represent 3 minutes of sound? (Without compression) </li></ul><ul><li>2) If we are representing speech phonemes, and we want to represent individual words, how much data does it take to encode the English word “Hello”? </li></ul><ul><ul><li>Assume that English has 39 phonemes </li></ul></ul><ul><ul><li>Assume that we have one phoneme per character </li></ul></ul><ul><ul><li>Assume that our coding scheme needs to accommodate arbitrary English words </li></ul></ul>
  28. 28. Example- Solution (1) <ul><li>44,100 samples/sec * 2 bytes/sample = 88, 200 bytes/sec </li></ul><ul><li>88, 200 bytes/sec * 60 sec/minute = </li></ul><ul><li>5,292,000 bytes/min </li></ul><ul><li>5, 292, 000 bytes/min * 3 min = </li></ul><ul><li>15,876,000 bytes </li></ul><ul><li>15,876,000 bytes/(1024 bytes/KB) = </li></ul><ul><li>15, 503.90625 KB </li></ul><ul><li>(Or about 15 MB) </li></ul>
  29. 29. Example- Solution (2) <ul><li>Problem is to represent sequences of phonemes </li></ul><ul><ul><li>Each phoneme can take on 39 possible values </li></ul></ul><ul><ul><li>Need at least 6 bits of information to represent 64 different values </li></ul></ul><ul><ul><ul><li>So, need at least 6 bits per phoneme </li></ul></ul></ul><ul><li>Word “Hello” has 5 characters </li></ul><ul><ul><li>Assuming also 5 phonemes </li></ul></ul><ul><ul><li>5 * 6 = 30 bits </li></ul></ul><ul><ul><li>30 bits/(8 bits/byte) = 3.75 bytes </li></ul></ul><ul><ul><li>Need 4 bytes </li></ul></ul>
  30. 30. Sound Filtering <ul><li>A sound can be filtered to remove a certain set of frequencies </li></ul><ul><li>Some types of filtering </li></ul><ul><ul><li>Low pass: Allows only low frequencies through </li></ul></ul><ul><ul><li>High pass: Allows only high through </li></ul></ul><ul><ul><li>Band pass: Allows only a band range of frequencies through </li></ul></ul><ul><li>Other types of filtering or processing </li></ul><ul><ul><li>Trimming, splicing, volume, brightness </li></ul></ul>
  31. 31. Java - 1 <ul><li>Sun Microsystems </li></ul><ul><li>Goal: Platform independent programs </li></ul><ul><li>Java code compiled into “byte-code” </li></ul><ul><ul><li>To be executed on a Java virtual machine </li></ul></ul><ul><ul><li>Really a byte-code interpreter </li></ul></ul><ul><ul><li>Executes individual byte-code commands </li></ul></ul><ul><ul><li>These are the files that are executed by browsers </li></ul></ul><ul><li>Some Java features </li></ul><ul><ul><li>No pointers </li></ul></ul><ul><ul><li>Garbage collection </li></ul></ul>
  32. 32. Java - 2 <ul><li>Efficiency </li></ul><ul><ul><li>Just-in-time compilers </li></ul></ul><ul><ul><ul><li>Compile byte-code to native machine code when used </li></ul></ul></ul><ul><ul><li>Java processing hardware </li></ul></ul><ul><li>Security </li></ul><ul><ul><li>Java code downloaded from the Internet is “untrusted” </li></ul></ul><ul><ul><li>Applet restrictions </li></ul></ul><ul><ul><li>Byte-code verification </li></ul></ul><ul><ul><ul><li>Byte-codes checked before running to ensure they are valid </li></ul></ul></ul><ul><ul><ul><li>Goal: ensure no unauthorized references to discs etc. </li></ul></ul></ul>
  33. 33. Web browsers <ul><li>Process various file types </li></ul><ul><ul><li>HTML </li></ul></ul><ul><ul><li>Media formats: images, sounds, movies </li></ul></ul><ul><li>Some types of data are processed directly </li></ul><ul><ul><li>Built into the browser </li></ul></ul><ul><li>Other types </li></ul><ul><ul><li>Require additional programs or “plugin-ins” </li></ul></ul><ul><ul><li>E.g., a plugin to display PDF or Adobe Portable Document format files or a plugin to display Shockwave files </li></ul></ul><ul><ul><li>Some plugins are called “players” or “viewers” </li></ul></ul><ul><ul><ul><li>E.g., a viewer to display MS Word documents </li></ul></ul></ul>
  34. 34. Kappner & Steinmetz (1995): Multimedia platform requirements: <ul><li>Multimedia platform should </li></ul><ul><ul><li>Support </li></ul></ul><ul><ul><ul><li>the development of a wide variety of multimedia applications </li></ul></ul></ul><ul><ul><ul><li>applications in a distributed system </li></ul></ul></ul><ul><ul><li>Be able to </li></ul></ul><ul><ul><ul><li>establish and control streams of continuous media </li></ul></ul></ul><ul><ul><ul><li>to choose the quality of service for a data stream </li></ul></ul></ul><ul><ul><ul><li>to express relations between several media streams </li></ul></ul></ul>
  35. 35. Multimedia Delivery Software <ul><li>Two general means of delivery </li></ul><ul><ul><li>Compiled or Interpreted </li></ul></ul><ul><li>Compiled </li></ul><ul><ul><li>Machine code generated for a computer CPU (e.g., Intel-based) and operating system (O/S) </li></ul></ul><ul><ul><li>Faster; limited to computer type & O/S; stand-alone </li></ul></ul><ul><ul><li>E.g., Macromedia Director “Projector” </li></ul></ul><ul><li>Interpreted </li></ul><ul><ul><li>Instructions created that can be interpreted by a program; E.g., a plugin </li></ul></ul><ul><ul><li>Slower; more portable; need interpreter (e.g., player) software </li></ul></ul><ul><ul><li>E.g., Macromedia Shockwave </li></ul></ul><ul><li>In both cases, media files are embedded and/or retained as separate files </li></ul>

×