Building Scalable and High Efficient Java Multimedia ...


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Building Scalable and High Efficient Java Multimedia ...

  1. 1. Building Scalable and High Efficient Java Multimedia Collaboration Wenjun Wu, Tao Huang, Geoffrey Fox Community Grids Computing Laboratory, Indiana University, USA
  2. 2. Global-MMCS & XGSP  XGSP • A common AV signaling protocol • A core conference control mechanism • Collaboration Web-Services  Global Multimedia Collaboration System (Global- MMCS) • to support scalable web-service based interoperable collaborations • integrates various services including videoconferencing, instant messaging and streaming, and supports multiple videoconferencing technologies such as H.323, SIP and Access Grid clients
  3. 3. Global-MMCS & XGSP Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing Session Server Media Servers XGSP-based Control Web Filters Services NaradaBrokering High Performance (RTP) NB Scales as All Messaging and XML/SOAP and .. distributed Admire SIP H323 Access Grid Native XGSP Gateways convert to uniform XGSP Messaging NaradaBrokering
  4. 4. Java Media Framework  Java Media Framework (JMF) • A unified framework for managing the acquisition, processing, and delivery of time-based media data • A common cross-platform Java API for accessing underlying media frameworks. • “Write Once, Run Anywhere” to multimedia developers
  5. 5. JMF for multimedia collaboration  JMF is the key building block for the implementation of Global-MMCS For both client-side and server-side Building multimedia services such as video, audio mixing, snapshot generation Implementing Global-MMCS AVportlet, a native XGSP audiovisual client  Performance Issues in JMF the performance optimization of Java Media Framework has to be made especially for the conferencing application
  6. 6. Multimedia Programming Frameworks Video Capture Video Audio Rendering Windows VFW or WDM Direct Draw Direct Sound Linux Video-for-Linux XLib ALSA Mac OS Sequence Grabber Quick Draw Core Audio Portability vs Performance : Platform-dependent multimedia devices and codec management a portable multimedia framework must define a good platform-independent part to cover the heterogeneous native multimedia platforms.
  7. 7. Multimedia Programming Frameworks  the similar multimedia filter pipeline architecture  Filter & Pipe design pattern cascaded filter chain  Support high efficient media processing • High-volume multimedia data is typically generated by a source filter objects and piped through one or more filter objects. Eventually, the media reaches a sink filter object and is consumed.
  8. 8. Multimedia Programming Frameworks  Tcl/Tk based CMT [7], VIC [8], VuSystem [9] low-overhead control functionality implemented in a scripting language like Tcl performance-critical data handling implemented in a compiled language like C/C++  Windows DirectShow C++ COM API only enables Windows applications for multimedia programming  JMF
  9. 9. RTP Vi deo RX RTP Vi deo Vi deo Sessi on Dat aSour ce Decoder Render er Manager M t or oni Net wor k RTP Audi o RX RTP Audi o Audi o Sessi on Dat aSour ce Decoder Render er Manager RTP Vi deo TX Vi deo Vi deo Raw Vi deo Sessi on Encoded Encoder Dat aSour ce Manager Dat aSour ce Net wor k RTP Audi o RX Audi o Audi o Raw Audi o Sessi on Encoded Encoder Dat aSour ce Manager Dat aSour ce JMF Filter Architecture
  10. 10. Access Grid Access Grid : a large scale audio/videoconference based on a multicast network  provides the group-to-group collaborations among 150 nodes connected to Internet 2 world wide.  Use improved MBONE audiovisual tools VIC and RAT  Depends upon high-speed network ( each node needs 20Mbps )
  11. 11. Issues for JMF in collaboration  Very good cross-platform framework  No optimization for conferencing application Compared to fully performance tuned Tcl/Tk package ( Access Grid )  No implementation for Mac OS
  12. 12. Enhanced Java Media Framework  Improving JMF performance by exposing the interfaces of the JMF filters and rewriting a faster one  Adding the state-of-art codecs in JMF and screen capture  Extending the JMF to the Mac-OS platform to extend its portability
  13. 13. Video Rendering  JMF filter pipeline is composed of filter Modules and linking Connectors  streaming protocols between a OutputConnector and InputConnector: Safe protocol Push protocol  JMF render implementation: safe protocol , blit the whole video frame  Our optimization approach: “Direct Write Through” approach
  14. 14. Direct-Write-Through optimization in JMF rendering Circular Output Buffer Input Connector Connector Direct Video Draw Decoder Renderer Mark OffScreen BLT Primary Data YUV Surface YUV Surface
  15. 15. Video Capturing  a video data source to abstract the real capture services which usually have different APIs in modern operating systems.  added Mac video capturing into JMF package  Added the desktop capture as a video data source to support remote desktop sharing
  16. 16. Add New Codec  JMF 2.0 package only supports H.261 decoder, H263 and JPEG. We added H.261 encoder and MPEG-4 video codec (DivX) based on JMF codec interface. In addition, new RTP format and payload for MPEG-4 video are also added through JMF RTPManager.
  17. 17. Global-MMCS AVPortlet V deo / A i udi o N ada ar V deo ocessor i Pr / Audi o Nonnect or C ada ar P ocessor r R /R P TP TC Connect or R S/ R on TP TC essi P R E TP vent S essi on V deo / A i udi o RTr E TPanspor t vent V deo P ayer o i /l Audi Tr anspor t P ayer l N ada ar B i dge r X S GP X S GP N adaB oker ar r A S V essi on A S V essi on JMS Cont r ol l er P ot ocol S ack r t Tr anspor t
  18. 18. Global-MMCS AVPortlet  GlobalMMCS AVPortlet, to fully make use of the services provided by Global-MMCS. The tool which can run on multiple desktop platforms such as windows, Linux and Mac OSX, integrates audio and video collaboration together  Extending JMF RTP Transport over Publish/Subscribe Overlay  XGSP Audiovisual Session Management
  19. 19. Performance Evaluation  Compare the video rendering performance between JMF and VIC Testing video streams are captured from the same desktop of the sending machine, and multicasted to the receiver machine which runs Access Grid VIC, two JMF clients based on Sun’s JMF implementation and our improved version.  two test scenarios with different source streams • the CIF-size still image of the desktop area is sent to the clients. Each stream is encoded in H.261, and has average bandwidth 20kbps. • the CIF-size video sequence from a 30-second movie with a lot of motions is streamed to the clients. Each stream is encoded in H.261, and has average bandwidth 500kbps.
  20. 20. Performance Evaluation 60 70 50 60 40 50 AG VIC AG VIC 40 30 SunJMF SunJMF 30 20 FastJMF FastJMF 20 10 10 0 0 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 Video Rendering performance (left: still desktop, right: movie sequence)
  21. 21. Q&A
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.