Milton Chen: Experiments in Desktop Videoconferencing


Published on

BayCHI June 14, 2005, program: Why is video conferencing not ubiquitous? Over 75 years after Bell Labs released the first videoconferencing system on April 7, 1927, we still do not experience videoconferencing as a common means to communicate today. Milton attributes the lack of adoption to limitations in technology and inadequate understanding of visual communication science.

Published in: Technology, Business
    Are you sure you want to  Yes  No
    Your message goes here
  • We just wanted to share information about best penis enlargement products that are busy talking about people who want a bigger penis size, stronger and able to increase stamina.
    Are you sure you want to  Yes  No
    Your message goes here
  • Vimax is the best enlargement pills for male virility enhancement, based on reviews from medical expert and Vimax customer testimonials.. Read full review at
    Are you sure you want to  Yes  No
    Your message goes here
  • Thanks for the review. I like your blog very much because it has very interesting articles of different topics like health and their tips and tricks. We glad to see you at safe ed.
    Are you sure you want to  Yes  No
    Your message goes here
  • Fioricet is often prescribed for tension headaches caused by contractions of the muscles in the neck and shoulder area. Buy now from and make a deal for you.
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Theme - Media for the masses Videoconferencing – how to make it much more common and ubiquitous; Podcasting Milton Chen – CTO VSee Lab BS Computer Science, UC Berkeley PHD in Electrical Engineering, Stanford University Thesis – Interactive distance learning Strong endorsements - Intel, Oracle Recognition/awards Colleagues - Erica ? Professor James Davis
  • Milton Chen: Experiments in Desktop Videoconferencing

    1. 1. Experiments in Videoconferencing Milton Chen CTO
    2. 2. The VSee Auditorium desktop interface 15’ x 5’ video wall
    3. 3. VSee <ul><li>2 nd place Stanford-Berkeley Innovator’s award </li></ul><ul><li>3 rd place Stanford business plan competition </li></ul><ul><li>Intel CEO Paul Otellini keynote </li></ul><ul><li>Oracle Executive VP Chuck Rozwat keynote </li></ul>Chuck Rozwat keynote
    4. 4. <ul><li>“ the breakthrough that collaboration gurus have been hunting for” - </li></ul><ul><ul><ul><ul><li>Jack Hirsch VP of Technology Shell </li></ul></ul></ul></ul>“ the world’s best videoconferencing system” - Cdr. Eric Rasmussen Iraq Humanitarian Operations Center Department of Defense <ul><li>“ uniquely suited for </li></ul><ul><li>planetwalk” - </li></ul><ul><ul><ul><ul><li>John Francis Goodwill Ambassador </li></ul></ul></ul></ul><ul><ul><ul><ul><li>United Nations </li></ul></ul></ul></ul>
    5. 5. What if there is no network infrastructure ?
    6. 6. Office of Secretary of Defense, State Department, NATO, United Nations … Strong Angel Kona, Hawaii 17-22 July 2004 VSee was selected as the real-time communication system
    7. 7. VSee at Strong Angel Provide global communication from a temporary shelter VIP presentation between Kona and DC
    8. 8. Ad-hoc peer-to-peer WiFi ~ 0.5 mile ~1 - 10 mile
    9. 9. Experiment 1: convoy protection VSee hops from car to car
    10. 10. Experiment 2: air-to-surface Can also airdrop arbitrary data setup screen shot
    11. 11. Experiment 3: ocean search and rescue
    12. 12. Experiment 3: ocean search and rescue The bottom video was from the live underwater camera held by the swimmer. The map with GPS annotation was shared using VSee setup screen shot
    13. 13. Experiment 3: ocean search and rescue
    14. 14. no pre-existing infrastructure <ul><li>VSee leverages what you have </li></ul><ul><ul><li>Internet </li></ul></ul><ul><ul><li>Internet2 </li></ul></ul><ul><ul><li>Satellite </li></ul></ul><ul><ul><li>WiMax </li></ul></ul><ul><ul><li>Cell phone </li></ul></ul><ul><li>VSee ad-hoc peer-to-peer WiFi </li></ul><ul><ul><li>Laptop + wireless card is all you need </li></ul></ul>
    15. 15. Afghanistan <ul><li>Visual fidelity comparable to high-end hardware </li></ul><ul><li>Secure (FIPS 140-2 and triple 256 bit AES ) </li></ul><ul><li>Never crash (59-day challenge) </li></ul><ul><li>Trivial to use (less than 60 seconds for 1 st time users) </li></ul>Kabul Nov 2004
    16. 16. March 2005 From VSee deployment team
    17. 17. VSee for tsunami relief UN headquarters in Jakarta
    18. 18. VSee in Darfur for refugee management CARE International field office Sudan, Africa
    19. 19. but
    20. 20. Why is videoconferencing not ubiquitous <ul><li>World’s first videoconferencing system </li></ul><ul><li>75 years later </li></ul><ul><ul><li>Technology limitations </li></ul></ul><ul><ul><li>Inadequate visual communication science </li></ul></ul>April 7, 1927 - Bell Labs 3x2 inch black&white display 1 msec end-to-end latency
    21. 21. VSee Peer-to-peer wireless How well can we judge eye contact “ The heart is stirred more slowly by the ear than by the eye .” – Horace
    22. 22. Eye contact stirs us to action [Sharbat Gula, photographed by McCurry ‘83]
    23. 23. Eye contact fires up our brain [Kampe et al. ’01 Nature ]
    24. 24. Eye contact sensitivity is high <ul><li>Spatial perception task </li></ul><ul><li>As good as Snellen acuity </li></ul>[Gibson and Pick ’63] 2 m * 6 observers judged 1 looker looker observer 0 8.5 -8.5 0 100 stdev = 2.8 ° Eye contact (%) Angle (deg)
    25. 25. Sensitivity is symmetric <ul><li>Cline ’67 </li></ul><ul><li>Kruger and Huckstedt ‘69 </li></ul><ul><li>Anstis, et al. ’69 </li></ul><ul><li>Stokes ’69 </li></ul><ul><li>Ellgring ’70 </li></ul>PicturePhone camera above display Hydra camera below display
    26. 26. Eye contact is difficult Looking into the camera Attempting eye contact
    27. 27. Solutions to eye contact Half-silvered mirror [Rosenthal ’47] MAJIC [Okada, et al. ’94] ClearBoard [Ishii, et al. ’92] GazeMaster [Gemmell, et al. ’00]
    28. 28. Methodology Observers watch videos of looker Large display with camera at the center
    29. 29. Eye contact?
    30. 30. Sensitivity is asymmetric * 16 observers judged recorded videos of 1 looker
    31. 31. An anatomical explanation looking at you looking sideways looking up looking down eye closing Illustrations from The Artist’s Guide to Facial Expression [Faigin ’90]
    32. 32. VSee Eye contact How well can we judge lip sync “ We shape our tools, and there after our tools shape us” - Marshal Mcluhan
    33. 33. Why read lips <ul><li>Improves comprehension </li></ul><ul><ul><li>Background noise [Sumby and Pollack ’54] </li></ul></ul><ul><ul><li>Hearing loss [Binnie, Montgomery, Jackson ’86] </li></ul></ul>[Yarbus ’ 67]
    34. 34. Audio ahead of the video <ul><li>Videoconferencing </li></ul><ul><ul><li>1 msec to encode audio </li></ul></ul><ul><ul><li>Up to 250 msec to encode MPEG-4 </li></ul></ul><ul><li>Detectable skew </li></ul><ul><ul><li>130 msec [Dixon and Spitz ’80] </li></ul></ul><ul><ul><li>80 msec [Steinmetz ’96] </li></ul></ul>
    35. 35. Conventional lip synchronization encode network decode A a v time Unsynchronized encode network decode sync a , v Audio delay line A delay skew
    36. 36. Attribute delay and skew to remote person <ul><ul><li>=> person is not believable? </li></ul></ul><ul><ul><li>=> person is slow? </li></ul></ul><ul><li> [Reeves and Nass ’96] </li></ul>encode network decode A a v time Unsynchronized encode network decode sync a , v Audio delay line A delay skew
    37. 37. A new lip sync method encode network decode sync synchronized and low perceived latency a v a v encode network decode A a v time Unsynchronized encode network decode sync a , v Audio delay line A Round trip delay
    38. 38. Methodology <ul><li>Recorded 3 speakers </li></ul><ul><ul><li>44.1KHz x 16 bps uncompressed audio </li></ul></ul><ul><ul><li>320x240x30fps uncompressed video </li></ul></ul><ul><ul><li>Sentences consist of easy to lipread words </li></ul></ul>Speaker 1 female native speaker Speaker 2 male native speaker Speaker 3 male non-native speaker
    39. 39. Perception of variable AV skew * 16 subjects judging recorded videos
    40. 40. VSee Eye contact Lip sync What frame rate is necessary “ We express ourselves into existence .” – Iris Murdoch
    41. 41. Minimum required frame rate <ul><li>Full motion 10-30 fps </li></ul><ul><li>Tolerable 5 fps </li></ul><ul><ul><li>[Tang and Isaac ’93] </li></ul></ul><ul><li>Lip synchronization 5 fps </li></ul><ul><ul><li>[Watson and Sasse ’96] </li></ul></ul><ul><li>Content understanding 5 fps </li></ul><ul><ul><li>[Ghinea and Thomas ’98] </li></ul></ul><ul><li>Sign language recognition 1 fps </li></ul><ul><ul><li>[Johnson and Caird ’96] </li></ul></ul>
    42. 42. Gesture Detection Algorithm input image frame difference after erosion Visualization of algorithm
    43. 43. Gesture sensitive transmission allows dynamic discussion 15 fps ~0.2 fps 0.2 fps * 8 groups of 4 people during a discussion * requires 10% of full motion bandwidth
    44. 44. Other studies [ Conveying Conversational Cues Through Video PhD Dissertation, 2003] When is a smile not a smile Value of f2f for discussion Visualizing the pulse of Classroom
    45. 45. VSee Eye contact Lip sync Gesture Telework “ Laugher is the shortest distance between two people ” – Victor Borge
    46. 46. VSee customers                                                                                        
    47. 47. telework => less money and influence <ul><li>Reasons to telework </li></ul><ul><li>Business continuity </li></ul><ul><li>Manage by results vs. time </li></ul><ul><li>… </li></ul><ul><li>No commute </li></ul><ul><li>Life style </li></ul><ul><li>… </li></ul><ul><li>but </li></ul><ul><li>no tool is able to bridge the physical distance </li></ul>
    48. 48. VSee Lab experiment <ul><li>Everybody works from home, </li></ul><ul><ul><li>hotels, cafes, libraries, airports, … since June 2003 </li></ul></ul><ul><ul><li>California, Michigan, Scotland, Taiwan, Malaysia </li></ul></ul><ul><li>Almost all customer interaction via VSee </li></ul><ul><li>Product support via desktop sharing </li></ul><ul><li>Product development via application sharing </li></ul><ul><li>Availability via presence indicator </li></ul>
    49. 49. Initial results <ul><li>What doesn’t work </li></ul><ul><ul><li>Still a sense of isolation </li></ul></ul><ul><ul><ul><li>Company meals and outings are critical! </li></ul></ul></ul><ul><ul><ul><li>Office of future will be social clubs? </li></ul></ul></ul><ul><ul><li>Remote whiteboard </li></ul></ul><ul><li>A surprising bonus </li></ul><ul><ul><li>Uninterrupted time to think </li></ul></ul><ul><ul><li>Building personal relationships </li></ul></ul>
    50. 50. I love to hear from you [email_address] Summary VSee Eye contact Lip sync Gesture Telework