Csc533 ch3a mm_framework


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Csc533 ch3a mm_framework

  1. 1. Multimedia Programming CSC533 Chapter 3a: Multimedia Frameworks 04/19/11 CSC533 - SAY
  2. 2. What is Software Framework? <ul><li>In computing , a software framework provides “the skeleton of an application that can be customized by an application developer” </li></ul><ul><li>Like software libraries , software frameworks aid the software developer by containing source code that solves problems for a given domain and provides a simple API . </li></ul>04/19/11 CSC533 - SAY
  3. 3. Types of software frameworks <ul><li>A software framework may focus on building graphical editors for different domains like artistic drawing, music composition, and mechanical CAD . </li></ul><ul><li>Another software framework can help build compilers for different programming languages and target machines. </li></ul><ul><li>Yet another might help build financial modeling applications or decision support systems </li></ul><ul><li>Frameworks exist for multimedia , web applications , and even communicating between different systems . </li></ul>04/19/11 CSC533 - SAY
  4. 4. What is Multimedia Framework? <ul><li>A multimedia framework (MMF) is a software framework that handles media on a computer and through a network. </li></ul><ul><li>A good multimedia framework offers an intuitive API and a modular architecture to easily add support for new codecs , container formats and transmission protocols . </li></ul><ul><li>It is meant to be used by applications such as media players and audio or video editors , but can also be used to build Videoconferencing applications , media converters and other multimedia tools. </li></ul><ul><ul><ul><ul><ul><li>Source from wikipedia. </li></ul></ul></ul></ul></ul>04/19/11 CSC533 - SAY
  5. 5. Multimedia Framework Requirements <ul><li>Economy of concepts </li></ul><ul><ul><li>MM framework should be based on small number of concepts, otherwise it becoming a maze </li></ul></ul><ul><ul><li>Identify any general concepts that apply across media types </li></ul></ul><ul><li>Open </li></ul><ul><ul><li>should be possible to extend MM framework to incorporated new media types, new data representations, new h/ware capabilities </li></ul></ul><ul><li>Queryable </li></ul><ul><ul><li>should specify interfaces for querying environments concerning their capabilities. Applications can then recognize missing functionality and adapt their behavior. </li></ul></ul>04/19/11 CSC533 - SAY
  6. 6. Multimedia Framework Requirements <ul><li>Distribution </li></ul><ul><ul><li>MM Framework should help applications in a way that facilitates distribution. In particular, objects within framework should correspond to easy-to-distribute units or subsystems. </li></ul></ul><ul><li>Scalable </li></ul><ul><ul><li>MM Framework should support scalable media representations. </li></ul></ul><ul><ul><li>Once media are in digital form, improvements in quality and capability are tied to advances in h/w </li></ul></ul><ul><ul><ul><li>Early version of processor might handle video at quarter screen resolution and few frames per second. </li></ul></ul></ul><ul><li>High-Level Interfaces </li></ul><ul><ul><li>MM Framework should provide high-level interfaces for media synchronization, media composition, device control, database integration and concurrent media processing activities. </li></ul></ul><ul><ul><li>Development is simplified if easy to use high-level interfaces are available </li></ul></ul>04/19/11 CSC533 - SAY
  7. 7. Multimedia Framework Example <ul><li>Consists of four main class hierarchies: (Click Figure) </li></ul><ul><li>Media classes </li></ul><ul><ul><li>Correspond to audio, video and the other media types. </li></ul></ul><ul><ul><li>Instances of these classes are particular media values (artifacts) </li></ul></ul><ul><li>Transform classes </li></ul><ul><ul><li>Represent media operations in a flexible and extensible manner. </li></ul></ul><ul><ul><li>For example, many image editing programs provide a large number of filter operations with which to transform images. These operations could be represented by methods of an image class. </li></ul></ul>04/19/11 CSC533 - SAY
  8. 8. Multimedia Framework Example <ul><li>Format classes </li></ul><ul><ul><li>Encapsulate information about external representations of media values. </li></ul></ul><ul><ul><li>Can be defined for both file formats: </li></ul></ul><ul><ul><ul><li>Image File format (such as GIF and TIFF) and ; </li></ul></ul></ul><ul><ul><ul><li>Uncompressed digital video “stream” formats (CCIR 601 4:2:2) </li></ul></ul></ul><ul><li>Component classes </li></ul><ul><ul><li>Represent h/w and s/w resources that produce, consume and transform media streams. </li></ul></ul><ul><ul><li>For instance, a CD-DA player is a component that produces a digital audio stream (specifically, stereo 16 bit PCM samples at 44.1 kHz). </li></ul></ul>04/19/11 CSC533 - SAY
  9. 9. 4 class hierarchies of a multimedia framework: 04/19/11 CSC533 - SAY BACK
  10. 10. Different Operating Systems and their Multimedia Support (Framework) <ul><li>Windows </li></ul><ul><ul><li>DirectX </li></ul></ul><ul><ul><li>.NET Framework </li></ul></ul><ul><li>Linux </li></ul><ul><ul><li>Gstreamer </li></ul></ul><ul><ul><li>SDL </li></ul></ul><ul><li>Mac OS X </li></ul><ul><ul><li>QuickTime </li></ul></ul><ul><li>Symbiam </li></ul><ul><ul><li>MMF </li></ul></ul><ul><ul><li>Helix </li></ul></ul><ul><li>Java </li></ul><ul><ul><li>JMF </li></ul></ul><ul><ul><li>JSRs </li></ul></ul>04/19/11 CSC533 - SAY
  11. 11. Windows : DirectX <ul><li>DirectX is the most important Multimedia Framework in Windows </li></ul><ul><li>First version was introduced in 1995 (Windows 95) </li></ul><ul><li>Current versions are 9.0 (Windows XP) and 10.0 (Windows Vista and Xbox 360) </li></ul>04/19/11 CSC533 - SAY
  12. 12. Windows : DirectX <ul><li>DirectX, formerly known as the Game SDK, is a term given for a collection of Windows APIs for easily handling tasks related to game programming Windows OS. </li></ul><ul><li>This set of development libraries for high performance games allows software developers &quot;direct access&quot; to the low-level functions of PC peripherals. </li></ul>04/19/11 CSC533 - SAY
  13. 13. Windows : DirectX APIs <ul><li>DirectX Graphics </li></ul><ul><ul><li>DirectDraw (2D) </li></ul></ul><ul><ul><li>Direct3D </li></ul></ul><ul><li>DirectInput </li></ul><ul><li>DirectPlay </li></ul><ul><li>DirectSound </li></ul><ul><ul><li>DirectSound3D </li></ul></ul><ul><li>DirectMusic </li></ul><ul><li>DirectSetup </li></ul><ul><li>DirectX Media </li></ul><ul><ul><li>DirectAnimation </li></ul></ul><ul><ul><li>DirectShow </li></ul></ul><ul><ul><li>DirectX Video Acceleration </li></ul></ul><ul><ul><li>Direct3D Retained Mode </li></ul></ul><ul><ul><li>DirectX Transform </li></ul></ul><ul><li>DirectX Media Objects </li></ul>04/19/11 CSC533 - SAY
  14. 14. Windows : . NET Framework <ul><li>Managed DirectX released to work with .NET </li></ul><ul><li>Is a software technology that is available with several Windows OS. </li></ul><ul><li>It includes a large library of pre-coded solutions to common programming problems, a runtime or virtual machine that manages the execution of programs written specifically for the framework , and a set of tools for configuring and building applications. </li></ul><ul><li>The .NET Framework is a key Microsoft offering and is intended to be used by most new applications created for the Windows platform. </li></ul>04/19/11 CSC533 - SAY
  15. 15. Linux : GStreamer <ul><li>GStreamer is a framework for creating streaming media applications. It can be used to implement any kind of multimedia application too. </li></ul><ul><li>The framework is based on plugins that will provide the various codec and other functionality. </li></ul><ul><li>GStreamer is used, e.g., in Maemo environment (Nokia tablets N800 and 770) and GNOME desktop environment </li></ul><ul><li> </li></ul>04/19/11 CSC533 - SAY
  16. 16. Linux : SDL <ul><li>Simple DirectMedia Layer (SDL) is an opensource, cross-platform multimedia library. </li></ul><ul><li>It gives access to audio, keyboard, mouse, joystick, 3D hardware (i.e., OpenGL), and 2D Video Frame buffer. </li></ul><ul><li>SDL supports Windows, Linux, Mac OS, etc. </li></ul><ul><li> </li></ul>04/19/11 CSC533 - SAY
  17. 17. Mac OS X : Quicktime <ul><li>QuickTime is Apple's multimedia framework. </li></ul><ul><li>Supports Mac OS X, Windows and old Mac OS. </li></ul><ul><li>API for encoding and decoding audio and video. </li></ul><ul><li>QuickTime file format is a multimedia container file. Competitors: MS ASF, ogg, Matroska. </li></ul><ul><li>QuickTime for Java supports capturing, editing, playback, etc. </li></ul><ul><li> </li></ul>04/19/11 CSC533 - SAY
  18. 18. Symbian : Multimedia Framework (MMF) <ul><li>Symbian OS was originally built for handheld devices, with limited resources. </li></ul><ul><li>A completely new Multimedia Framework for Symbian was introduced in Symbian 7.0s </li></ul><ul><li>MMF provides a lightweight, multi-threaded framework for handling multimedia data. </li></ul><ul><li>MMF enables: </li></ul><ul><ul><li>Audio recording, playback, and streaming </li></ul></ul><ul><ul><li>Video recording, playback, and streaming </li></ul></ul><ul><ul><li>Image related functionalities </li></ul></ul><ul><li>MMF allows developers to write efficient and powerful plug-ins </li></ul>04/19/11 CSC533 - SAY
  19. 19. Symbian : Helix Framework <ul><li>Helix DNA is an open-source multimedia player project based on the RealPlayer. </li></ul><ul><li>The Helix Framework contains streaming server, players, codecs, and production tools. </li></ul><ul><li>Since May 2006, Helix was included in over a hundred mobile phones and was distributed more than 125 million times. It is also distributed within Maemo environment. </li></ul>04/19/11 CSC533 - SAY
  20. 20. Java : Java Media Framework (JMF) <ul><li>JMF allows use of multimedia in Java applications </li></ul><ul><li>Real-time network protocols </li></ul><ul><li>Multiplexing </li></ul><ul><li>Codecs </li></ul><ul><li>Players </li></ul><ul><li>Effects </li></ul><ul><li>Capture </li></ul><ul><li>Control </li></ul>04/19/11 CSC533 - SAY
  21. 21. Java : Java Specification Request (JSR) <ul><li>Third parties can participate in Java platform development by proposing JSRs. </li></ul><ul><li>A Java Specification Request (JSR) describes a technology to be added to the Java platform. </li></ul><ul><li>Most important Multimedia-related JSRs are: </li></ul><ul><ul><li>JSR 135: Java Mobile Media API (for J2ME platform) </li></ul></ul><ul><ul><li>JSR 32 and JSR 180: Java SIP APIs </li></ul></ul><ul><ul><li>JSR 231 and 926: Java 3D graphic APIs </li></ul></ul><ul><ul><li>JSR 184 and 239: Java 3D graphic APIs for J2ME </li></ul></ul><ul><ul><li>JSR 226: Scalable 2D Vector Graphics API (J2ME) </li></ul></ul><ul><ul><li>JSR 234: Java Advanced Multimedia API for J2ME </li></ul></ul><ul><ul><li>JSR 272: Mobile Broadcast Service API for J2ME </li></ul></ul>04/19/11 CSC533 - SAY