Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hello 1 2 3, can you see me now?

2,568 views

Published on

Learning from the mistakes of the past and knowing where we stand at present will help us build the Internet video communication systems of the future. I present my point of view on the evolution, challenges and mistakes of the past, and, moving forward, describe the challenges in bridging the gap between web and VoIP. I highlight my contributions at various stages in the journey of Internet audio/video communication protocols.

Published in: Technology

Hello 1 2 3, can you see me now?

  1. 1. Internet video communication: past, present and future Hello 1 2 3, can you hear me now? © 2011, Kundan Singh, http://kundansingh.com Kundan Singh, PhD Nov 2011 see
  2. 2. Topics Video over phone lines IP multicast Voice (+video) over IP (interactive) video streaming Web + VoIP past future + challenges; my contributions
  3. 3. 1997 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Undergrad @India MS @columbia [email_address] @Motorola H.323 video client gateway SIP-H.323 translator SIP-RTSP voice mail SIP conferencing Libsip++ (SIP library) P2P VoIP using SIP SIP Failover/load sharing Enterprise VoIP infrastructure Interactive voice response CINEMA user interface Multimedia collaboration Mobile NAT Reliability & scalability VoIP infrastructure Multiparty conference performance My Background Industry Adobe, Tokbox, 6Connex, Twilio, and Open Source Add SIP to Flash Player Pubic Flash video conference Enterprise video conference Cloud telephony Web video communication
  4. 4. What is Internet video communication? “ Devil is in the details” echo cancellation, video layout, bandwidth reservation, error concealment, … protocol for transfer, scalability, robustness, session negotiation, … … … The Internet
  5. 5. Video over Phone Lines Too complex Telecom DNA My first video telephony project IP and lower layers TCP UDP TPKT Q.931 H.245 RAS RTCP RTP Codecs Terminal Control/Devices Q.931 SETUP Q.931 CONNECT Terminal Capabilities Terminal Capabilities Open Logical Channel Open Logical Channel H.323
  6. 6. Once you have the hammer, everything looks like a nail
  7. 7. IP Multicast Encoded Audio/Video RTP Header UDP header IP header
  8. 8. Reality check: no multicast; plagued with middle foxes boxes
  9. 9. Voice over IP <ul><li>Internet engineers vs telecom DNA </li></ul><ul><li>Embrace HTTP, email </li></ul><ul><li>SIP is simple </li></ul><ul><li>Built many apps </li></ul>+ Video ⁁ was back then
  10. 10. “ SIP has become the SS8 of PSTN trunking” -- Jiri Kuthan Asserted identity GRUU Authenticated identity management Reliability of early media Update session for early media PINT service protocol Non-adjacent contacts SPIRITS call waiting INFO for ISUP SIP to ISUP SIP to QSIG ISUP overlapped signaling in SIP Service discovery on Path Request history of network Call transfer Join-Replaces Publish state Conditional event notification Presence event package <ul><li>Session initiation vs application logic </li></ul><ul><li>Telecom apps sidelined Internet apps </li></ul><ul><li>New “managed” services appeared </li></ul><ul><li>Business interest overshadowed research </li></ul>Real Internet Problems NAT traversal End-to-end services Security
  11. 11. RFC 5638 Simple SIP usage scenario for applications in the endpoints Breaking Free P P P P P
  12. 12. Video Streaming Interactive
  13. 13. Why is this approach promising?
  14. 14. almost ⁁ Available to everyone Easy for developers Immersive experience No download needed additional ⁁
  15. 15. <ul><li>What did we learn? </li></ul><ul><ul><ul><ul><li>“ just works” = happy users </li></ul></ul></ul></ul><ul><ul><ul><ul><li>“ quick/simple” = happy developers </li></ul></ul></ul></ul><ul><ul><ul><ul><li>There is no third rule </li></ul></ul></ul></ul>
  16. 16. alice bob
  17. 17. NetConnection Camera Microphone Video VideoDisplay NetStream preview publish(“mine”) play(“yours”) create
  18. 18. Flash-VideoIO <object id= &quot;video1&quot; width=&quot;320&quot; height=&quot;240“ ... > <param name=&quot;movie&quot; value= &quot;VideoIO.swf&quot; /> ... <param name=&quot; flashVars &quot; value=&quot;...&quot; /> <embed src= &quot;VideoIO.swf&quot; width=&quot;320&quot; height=&quot;240&quot; name= &quot;video1&quot; flashVars =&quot;...&quot; ... > </embed> </object> <script> getFlashMovie( &quot;video1&quot; ).setProperty( &quot;src&quot; , &quot;rtmp://server/path/123?publish=alice&quot; ); </script> flashVars=“controls=true” NetConnection Camera Microphone Video VideoDisplay NetStream
  19. 19. Flash-VideoIO <object id= &quot;video1&quot; width=&quot;320&quot; height=&quot;240“ ... > <param name=&quot;movie&quot; value= &quot;VideoIO.swf&quot; /> ... <param name=&quot; flashVars &quot; value=&quot;...&quot; /> <embed src= &quot;VideoIO.swf&quot; width=&quot;320&quot; height=&quot;240&quot; name= &quot;video1&quot; flashVars =&quot;...&quot; ... > </embed> </object> <script> getFlashMovie( &quot;video1&quot; ).setProperty( &quot;src&quot; , &quot;rtmp://server/path/123?publish=alice&quot; ); </script> flashVars=“controls=true”
  20. 21. Multi-party audio/video conference http://public-chat.appspot.com/ < 1000 lines of code
  21. 22. siprtmp.py in action
  22. 23. “All that glitters is not gold”
  23. 24. Latency in media path proprietary not always RTMP End to end media RTMFP media
  24. 25. Echo cancellation High quality encoder missing not integrated
  25. 26. Separate islands of innovations VoIP Web
  26. 27. <ul><li>Provider and user perspective </li></ul><ul><li>Trust model, session negotiation, … </li></ul>From VoIP to Web
  27. 28. <ul><li>HTTP </li></ul><ul><li>signaling and control </li></ul><ul><li>UDP </li></ul><ul><li>media transport </li></ul>Minimum Protocol Requirements
  28. 29. <ul><li>Modify browser </li></ul><ul><li>Extend web protocols/languages </li></ul><ul><li>Include SIP/RTP stack </li></ul><ul><li>Add device access, codecs and e2e transport (IETF, W3C) </li></ul>Use existing plugin Most existing web communication systems use Flash Player HTTP UDP proprietary proprietary over UDP Web server server HTML HTML HTML HTML FP FP Available Options Build new plugin Just handles missing pieces (device access, codec, transport) HTTP UDP Web server HTML HTML plugin plugin Use separate app Runs as a separate process/service HTTP UDP Web server HTML App App HTML HTML
  29. 30. Modify browser No other dependencies Eventually a standard Numerous web developers Reluctance to change Portable device access/sharing Time to ubiquitous availability Use existing plugin Ubiquitous availability Browser agnostic Rich developer tools and experience One-to-one as well as group Transport is not enough (for SIP/RTP) Cannot install new codecs Depends on vendor for updates Comparison Use separate app Browser and app agnostic Any transport, language, codecs. Persistent/long lived state Yet another install, slow adoption Security and access control Video display needs plugin Available Options <ul><li>With existing technologies </li></ul><ul><li>Emerging standard protocols </li></ul><ul><li>Allow walled garden </li></ul><ul><li>Require new install </li></ul><ul><li>App dies on page close </li></ul><ul><li>Re-use web security means </li></ul>
  30. 31. Voice and Video on Web DB Apache web server, PHP websocket server, MySQL database <ul><li>Signaling API </li></ul><ul><li>Resource-based, SDP=>XML/JSON, </li></ul><ul><li>subscribe/notify, long-lived connection, </li></ul><ul><li>persistent vs transient data, access control </li></ul>(2) Communication Widgets Click to call, contact list, conference object Project: http://sites.google.com/site/vvowproject/ Source: http://code.google.com/p/vvowproject/ Demo: http://gardo1.rice.iit.edu/webconf/ Applications >> Protocols (3) Media Application API Transport, auth and media objects
  31. 32. Summary Video over phone lines IP multicast Voice (+video) over IP (interactive) video streaming Web + VoIP past future
  32. 33. <ul><li>Stay technical; R&D </li></ul><ul><li>Web RTC, but more on tech demos </li></ul><ul><li>Global cloud-based SIP services </li></ul><ul><li>Mobile and custom video devices </li></ul>Moving Forward http://myprojectguide.org

×