Introduction  Principles of Multimedia Systems Sethserey SAM Computer Science Department
`
Multimedia System <ul><li>Multimedia involves more than simple addition of new data types. </li></ul><ul><li>It integrates...
Challenges of Multimedia Computing <ul><li>Developing a successful multimedia system is non-trivial. </li></ul><ul><ul><li...
Multimedia is Multidisciplinary Image, audio, speech processing Computer vision, pattern recognition Computer networks, op...
Example Multimedia Systems  Real Estate  in MIT’s Project Athena
Example Multimedia Systems Chronoscope  in MIT’s Project Athena
Example Multimedia Systems Navigation   Learning Environment  in MIT’s Project Athena
Example Multimedia Systems Photosynth of Microsoft Live Labs.
Multimedia Computing <ul><li>Multimedia systems involve some basic enabling techniques: </li></ul><ul><ul><li>Multimedia d...
Data Compression Raw image takes about 6M bytes (without header information) 24k bytes with jpeg, Q=50
“JPEG” Illustration r g b Transform to frequency domain Quantization Lossless compression … 0010001 …. decoder r’ g’ b’ u v
Media Processing and Analysis <ul><li>In applications such as digital library, automatic data analysis has to be done to e...
Tracking Object by Image Matching
Multiple Object Tracking  Toys Squash Lab Double Squash
Finding Objects in Clutter
Object Recognition David G. Lowe,  &quot;Object recognition from local scale-invariant features,&quot;  ICCV99.
Posture Detection in Images Template Image
Dealing with Multiple Objects
 
Finding Action Clusters in Image Database
Finding Action Clusters (con’t)
Action Recognition video1 video2 Ramanan, D., Forsyth, D. A., Zisserman, A. &quot; Strike a Pose: Tracking People by Findi...
Media Delivery <ul><li>Transmitting multimedia data across the network is another topic in multimedia computing. </li></ul...
Multimedia Database and Indexing <ul><li>Multimedia database has to deal with large media files. </li></ul><ul><li>Multime...
Useful Resources <ul><li>Journals </li></ul><ul><ul><li>IEEE Multimedia </li></ul></ul><ul><ul><li>IEEE Transaction on Mul...
Q & A
Upcoming SlideShare
Loading in...5
×

Introduction

533

Published on

Multimedia

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
533
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Introduction

  1. 1. Introduction Principles of Multimedia Systems Sethserey SAM Computer Science Department
  2. 2. `
  3. 3. Multimedia System <ul><li>Multimedia involves more than simple addition of new data types. </li></ul><ul><li>It integrates a wide range of symbol modes simultaneously into a coherent framework. </li></ul><ul><li>The framework is usually denoted as a multimedia system. </li></ul>
  4. 4. Challenges of Multimedia Computing <ul><li>Developing a successful multimedia system is non-trivial. </li></ul><ul><ul><li>Continuous media types such as video need a lot of space to store and very high bandwidth to transmit. </li></ul></ul><ul><ul><li>They also have tight timing constraints. </li></ul></ul><ul><ul><li>Automatically analyzing, indexing and organizing information in audio, image and video is much harder than from text. </li></ul></ul><ul><ul><li>Multimedia involves many different research areas and needs more complex and more efficient algorithms and hardware platforms. </li></ul></ul>
  5. 5. Multimedia is Multidisciplinary Image, audio, speech processing Computer vision, pattern recognition Computer networks, operating system Human computer interaction Computer graphics Multimedia computing
  6. 6. Example Multimedia Systems Real Estate in MIT’s Project Athena
  7. 7. Example Multimedia Systems Chronoscope in MIT’s Project Athena
  8. 8. Example Multimedia Systems Navigation Learning Environment in MIT’s Project Athena
  9. 9. Example Multimedia Systems Photosynth of Microsoft Live Labs.
  10. 10. Multimedia Computing <ul><li>Multimedia systems involve some basic enabling techniques: </li></ul><ul><ul><li>Multimedia data representation and compression. </li></ul></ul><ul><ul><li>Multimedia data processing and analysis. </li></ul></ul><ul><ul><li>Transmitting multimedia data through communication networks. </li></ul></ul><ul><ul><li>Multimedia database, indexing and retrieval. </li></ul></ul>
  11. 11. Data Compression Raw image takes about 6M bytes (without header information) 24k bytes with jpeg, Q=50
  12. 12. “JPEG” Illustration r g b Transform to frequency domain Quantization Lossless compression … 0010001 …. decoder r’ g’ b’ u v
  13. 13. Media Processing and Analysis <ul><li>In applications such as digital library, automatic data analysis has to be done to extract semantic meanings from audios, images and videos. </li></ul><ul><li>Based on media processing methods such as </li></ul><ul><ul><li>object tracking (face, eyes), </li></ul></ul><ul><ul><li>object recognition, </li></ul></ul><ul><ul><li>gesture recognition, etc., </li></ul></ul><ul><li>we can build more effective human computer interfaces. </li></ul>
  14. 14. Tracking Object by Image Matching
  15. 15. Multiple Object Tracking Toys Squash Lab Double Squash
  16. 16. Finding Objects in Clutter
  17. 17. Object Recognition David G. Lowe, &quot;Object recognition from local scale-invariant features,&quot; ICCV99.
  18. 18. Posture Detection in Images Template Image
  19. 19. Dealing with Multiple Objects
  20. 21. Finding Action Clusters in Image Database
  21. 22. Finding Action Clusters (con’t)
  22. 23. Action Recognition video1 video2 Ramanan, D., Forsyth, D. A., Zisserman, A. &quot; Strike a Pose: Tracking People by Finding Stylized Poses. &quot; Computer Vision and Pattern Recognition (CVPR), San Diego, CA, June 2005. video3
  23. 24. Media Delivery <ul><li>Transmitting multimedia data across the network is another topic in multimedia computing. </li></ul><ul><li>There are different issues when we transmit video and audio through packet network. </li></ul><ul><ul><li>Quality of service. </li></ul></ul><ul><ul><li>Synchronization. </li></ul></ul><ul><ul><li>Error and congestion control. </li></ul></ul><ul><ul><li>Session setup and book keeping. </li></ul></ul>
  24. 25. Multimedia Database and Indexing <ul><li>Multimedia database has to deal with large media files. </li></ul><ul><li>Multimedia data needs new data structures, indexing and searching methods. </li></ul><ul><li>Content based multimedia retrieval is still an ongoing research topic. </li></ul>
  25. 26. Useful Resources <ul><li>Journals </li></ul><ul><ul><li>IEEE Multimedia </li></ul></ul><ul><ul><li>IEEE Transaction on Multimedia </li></ul></ul><ul><ul><li>IEEE Transaction on Image Processing </li></ul></ul><ul><ul><li>IEEE Transaction on Pattern Recognition and Machine Intelligence </li></ul></ul><ul><li>Conferences </li></ul><ul><ul><li>ACM Multimedia </li></ul></ul><ul><ul><li>International Conference on Multimedia and Expo (ICME) </li></ul></ul><ul><ul><li>IEEE Computer Vision and Pattern Recognition (CVPR) </li></ul></ul><ul><ul><li>International Conference on Computer Vision (ICCV) </li></ul></ul>
  26. 27. Q & A
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×