Developing Cross-Platform  Audio and Music Applications with the CLAM Framework  ACM Multimedia '06 <ul><ul><li>Xavier Ama...
Index <ul><li>Introduction </li></ul><ul><li>Applications </li></ul><ul><li>Infrastructure </li></ul><ul><li>Repositories ...
<ul><li>Started in October 2000 </li></ul><ul><li>There are more than 250 C++ classes (50.000 loc), compiled under GNU/ Li...
Highlights <ul><li>Although it currently specializes in audio and music, it presents a metamodel that has proven valid for...
Applications. Annotator
Applications. SMSTools
Applications. Network Editor
Applications. Others
CLAM Components
Infrastructure. CLAM Network model <ul><li>The CLAM network is a graphical model of computation based on Dataflow Process ...
Infrastructure. Ports,  Controls, and Configurations <ul><li>Two kinds of connection mechanisms between Processing Objects...
Infrastructure. Processing
Infrastructure. Dynamic vs. Static Compositions <ul><li>Processing Objects can be grouped using two different mechanisms: ...
Repositories (black-box) <ul><li>Ready-to-use  processing classes  (almost 150): </li></ul><ul><ul><li>Analysis (FFT, spec...
Tools. Platform Abstraction <ul><li>Audio I/O </li></ul><ul><li>MIDI I/O </li></ul><ul><li>Audio File I/O: wav, aiff, mp3,...
Tools <ul><li>XML </li></ul><ul><li>Any Processing Data or Configuration has automatic  XML persistency. </li></ul><ul><li...
External Open Source libraries <ul><li>FFTW (FFT) </li></ul><ul><li>Xercesc & libxml (XML using DOM API) </li></ul><ul><li...
Conclusions <ul><li>CLAM is currently in release 0.91 and we plan on getting to 1.0 by the end of 2006. </li></ul><ul><li>...
Demo
Visit us at.... www.clam.upf.es Xavier Amatriain, Pau Arumi,  David Garcia ...
Upcoming SlideShare
Loading in …5
×

The CLAM Framework

1,657 views

Published on

Presentation on the CLAM Framework that we used in ACM Multimedia 2006 when we were given the Award to the Best Open Source Multimedia Software.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,657
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The CLAM Framework

  1. 1. Developing Cross-Platform Audio and Music Applications with the CLAM Framework ACM Multimedia '06 <ul><ul><li>Xavier Amatriain (UCSB) </li></ul></ul><ul><ul><li>Pau Arumí (UPF) </li></ul></ul><ul><ul><li>David Garcia (UPF) </li></ul></ul>
  2. 2. Index <ul><li>Introduction </li></ul><ul><li>Applications </li></ul><ul><li>Infrastructure </li></ul><ul><li>Repositories </li></ul><ul><li>Tools </li></ul><ul><li>Conclusions </li></ul><ul><li>Demo! </li></ul>
  3. 3. <ul><li>Started in October 2000 </li></ul><ul><li>There are more than 250 C++ classes (50.000 loc), compiled under GNU/ Linux, Windows and OSX. </li></ul><ul><li>CLAM has been used for various audio projects: time-stretching, real-time sax synthesis, content analysis and Mpeg7 description, real-time audio effects, audio annotation... </li></ul><ul><li>It has also been used for educational purposes. </li></ul>Factsheet
  4. 4. Highlights <ul><li>Although it currently specializes in audio and music, it presents a metamodel that has proven valid for general multimedia. </li></ul><ul><li>Deals well with spectral-domain processing </li></ul><ul><li>Can process streams with different kinds of data types, even user-defined </li></ul><ul><li>Two different working modes: application framework and rapid-prototyping . </li></ul><ul><li>It is really Object-oriented and has been documented through a Pattern Language </li></ul><ul><li>It is efficient and can be used for real-time applications </li></ul>
  5. 5. Applications. Annotator
  6. 6. Applications. SMSTools
  7. 7. Applications. Network Editor
  8. 8. Applications. Others
  9. 9. CLAM Components
  10. 10. Infrastructure. CLAM Network model <ul><li>The CLAM network is a graphical model of computation based on Dataflow Process Networks </li></ul><ul><li>Scheduling can be performed both statically and dynamically, depending on the particular application. </li></ul>
  11. 11. Infrastructure. Ports, Controls, and Configurations <ul><li>Two kinds of connection mechanisms between Processing Objects: </li></ul><ul><li>Ports </li></ul><ul><ul><li>Data is transmitted synchronously through FIFO queues </li></ul></ul><ul><li>Controls </li></ul><ul><ul><li>Events are transmitted asynchronously </li></ul></ul><ul><li>Configurations , unlike Controls, hold parameters that produce expensive or structural changes in a Processing Object </li></ul><ul><ul><li>They can only be set when the Processing Object is not running </li></ul></ul>
  12. 12. Infrastructure. Processing
  13. 13. Infrastructure. Dynamic vs. Static Compositions <ul><li>Processing Objects can be grouped using two different mechanisms: </li></ul><ul><li>Networks </li></ul><ul><ul><li>Created dynamically at run-time </li></ul></ul><ul><ul><li>Strive for flexibility and understandability </li></ul></ul><ul><ul><li>Offer automatic flow and data management </li></ul></ul><ul><li>Processing Composites </li></ul><ul><ul><li>Created statically at compile-time </li></ul></ul><ul><ul><li>Strive for efficiency </li></ul></ul><ul><ul><li>The programmer is in charge of most flow management issues. </li></ul></ul>
  14. 14. Repositories (black-box) <ul><li>Ready-to-use processing classes (almost 150): </li></ul><ul><ul><li>Analysis (FFT, spectral analysis, SMS analysis, Tonal Analysis, descriptor extraction...), Arithmetic Operators, Input/Output Processing Objects (Audio, AudioFile, MIDI, SDIF), Generators, Transformation, Synthesis </li></ul></ul><ul><li>Ready-to-use data classes : </li></ul><ul><ul><li>Audio, Spectrum, SpectralPeakArray, Fundamental, Frame, Segment, Descriptors... </li></ul></ul>
  15. 15. Tools. Platform Abstraction <ul><li>Audio I/O </li></ul><ul><li>MIDI I/O </li></ul><ul><li>Audio File I/O: wav, aiff, mp3, ogg... </li></ul><ul><ul><li>ID3 tags. </li></ul></ul><ul><li>SDIF File support </li></ul><ul><li>Support for OSC, JACK, LADSPA, SDIF, VST, ASIO... </li></ul>
  16. 16. Tools <ul><li>XML </li></ul><ul><li>Any Processing Data or Configuration has automatic XML persistency. </li></ul><ul><li>GUI </li></ul><ul><li>Visualization module based on the Qt toolkit </li></ul><ul><li>Plus many ready-to-use graphical components (widgets) </li></ul><ul><li>Including debugging tools </li></ul>
  17. 17. External Open Source libraries <ul><li>FFTW (FFT) </li></ul><ul><li>Xercesc & libxml (XML using DOM API) </li></ul><ul><li>FLTK and QT GUI toolkits </li></ul><ul><li>RtAudio, PortAudio or DirectX (for Windows audio) </li></ul><ul><li>Libsndfile, Ogg-Vorbis, libmad (mp3), id3lib, for handling audio files. </li></ul><ul><li>oscpack </li></ul><ul><li>libjack </li></ul><ul><li>CppUnit (testing framework, only used for development) </li></ul><ul><li>pthreads (multithreading on Windows) </li></ul>
  18. 18. Conclusions <ul><li>CLAM is currently in release 0.91 and we plan on getting to 1.0 by the end of 2006. </li></ul><ul><li>But it already offers: </li></ul><ul><ul><li>A powerful framework that can yield efficient and robust applications. </li></ul></ul><ul><ul><li>A prototyping tool for rapid development </li></ul></ul><ul><ul><li>A number of ready-to-use applications. </li></ul></ul>
  19. 19. Demo
  20. 20. Visit us at.... www.clam.upf.es Xavier Amatriain, Pau Arumi, David Garcia ...

×