Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

cise-pi.ppt

367 views

Published on

  • Be the first to comment

  • Be the first to like this

cise-pi.ppt

  1. 1. Pervasive Pixels (Columbia University Dept. of Computer Science) Henning Schulzrinne (PI) Steven K. Feiner Gail Kaiser John Kender Kathleen McKeown
  2. 2. Proposed Research <ul><li>Goal: seamless mobile multimedia collaboration across distance </li></ul><ul><li>Integrate advances across fields </li></ul><ul><ul><ul><li>Collaborative work </li></ul></ul></ul><ul><ul><ul><li>Graphical and visual interfaces </li></ul></ul></ul><ul><ul><ul><li>Spoken language understanding and generation </li></ul></ul></ul><ul><ul><ul><li>Vision sensing and understanding </li></ul></ul></ul><ul><ul><ul><li>Networking and security </li></ul></ul></ul>
  3. 3. Contributions <ul><li>Contextual information management </li></ul><ul><ul><ul><li>use workflow to determine display content </li></ul></ul></ul><ul><ul><ul><li>multimedia summaries of past and present sessions </li></ul></ul></ul><ul><li>Harmonizing physical and virtual environments </li></ul><ul><ul><ul><li>map changing virtual information onto physical displays </li></ul></ul></ul><ul><ul><ul><li>map layout of physical environment onto virtual space </li></ul></ul></ul><ul><li>Network services </li></ul><ul><ul><ul><li>clear, flexible interface to common services </li></ul></ul></ul><ul><ul><ul><li>authentication and privacy support </li></ul></ul></ul><ul><ul><ul><li>infrastructure for persistent large displays </li></ul></ul></ul>
  4. 4. Features of Research Infrastructure <ul><li>Large numbers of instrumented multi-display workspaces </li></ul><ul><li>Networked mobile devices of various capabilities </li></ul><ul><li>Transparent and automatic adaptability to changes of place, platform or group </li></ul><ul><li>Support for a wide range of hardware and software, from commercial to novel </li></ul>
  5. 5. Proposed Research Infrastructure <ul><li>Outfit informal areas for collaboration </li></ul><ul><ul><ul><li>Public areas for walk-by interaction </li></ul></ul></ul><ul><ul><ul><li>Multiple touch displays, cameras, audio </li></ul></ul></ul><ul><ul><ul><li>Portable units </li></ul></ul></ul><ul><li>Stationary setups </li></ul><ul><ul><ul><li>Multiple displays, video cameras, audio </li></ul></ul></ul><ul><ul><ul><li>Seminar room, meeting rooms </li></ul></ul></ul><ul><ul><ul><li>12 faculty offices </li></ul></ul></ul><ul><li>User-based personalization: user location </li></ul><ul><ul><ul><li>Triangulation on mobile devices </li></ul></ul></ul><ul><ul><ul><li>Visual tracking </li></ul></ul></ul><ul><ul><ul><li>Standard methods (e.g., active badge) </li></ul></ul></ul>
  6. 6. Public areas – walk by stations <ul><li>Multiple touch displays, video projectors and cameras, embedded computers, speakers and microphones </li></ul>
  7. 7. IR/RF badge network PC proj. camera card reader loudspeaker ceiling electronic whiteboard microphone array Design for Walk-by Collaboration Station
  8. 8. Public Areas – informal gatherings
  9. 9. Meeting Room <ul><li>Remote-controlled pan-tilt video cameras and projectors, Omnicam, conference table microphones, automatic audio mixer, ceiling speakers </li></ul>
  10. 10. Faculty Office <ul><li>Mimio electronic whiteboard, XGA video projector, Ethernet speaker phone, wall-mounted pan-tilt video camera, PocketPCs </li></ul>
  11. 11. Seminar room <ul><li>Omnicam omnidrectional audience camera, high-resolution DV video camera, 2 pan-tilt speaker cameras, ceiling mounted microphones, electronic whiteboard, XGA high-brightness video projectors </li></ul>
  12. 12. Functionalities <ul><li>Conferencing </li></ul><ul><ul><ul><li>Internet conferencing server to mix IP and PSTN audio streams </li></ul></ul></ul><ul><li>Interconnection with analog phone </li></ul><ul><ul><ul><li>Digital hybrid connects digital or analog sound to existing telephone system in classroom </li></ul></ul></ul><ul><ul><ul><li>Network voice-over-IP interface attached to Nortel Meridian PBX for 20 simultaneous conversations </li></ul></ul></ul><ul><li>Multi-processor servers and IA64 compute and database server </li></ul><ul><li>File storage </li></ul><ul><li>Face, speaker and fingerprint recognition </li></ul><ul><li>Backup facilities: 2 printers and tape library system </li></ul>
  13. 13. Initial results <ul><li>HCI: gesture-based user interface for public kiosk </li></ul><ul><ul><li>mouse replacement for pointing and selecting </li></ul></ul><ul><ul><li>uses frontal and side camera </li></ul></ul><ul><li>Security: disCFS and WebDAVA secure file systems </li></ul><ul><ul><li>disCFS: NFS with credentials instead of authorization </li></ul></ul><ul><ul><li>WebDAVA: grant restricted access to resources using HTTP and Java applets </li></ul></ul><ul><li>Web-based collaboration: </li></ul><ul><ul><li>content on all kinds of devices </li></ul></ul><ul><ul><li>pass DOM through a series of filters and transformations  HTML </li></ul></ul><ul><li>Ubiquitous multimedia communications infrastructure </li></ul><ul><ul><li>being commercialized; I2 demonstration </li></ul></ul><ul><ul><li>input into standardization (IETF) </li></ul></ul>
  14. 14. Ubiquitous Computing <ul><li>Traditionally, focus on closed environments </li></ul><ul><ul><li>proprietary protocols </li></ul></ul><ul><ul><li>single (trusted) user class </li></ul></ul><ul><ul><li>single site (room, lab, home, …) </li></ul></ul><ul><ul><li>stand-alone components (“video conferencing”) </li></ul></ul><ul><li>PP focuses on whole system and user experience </li></ul><ul><li>Pervasive Pixels networking component: </li></ul><ul><ul><li>standard protocols: </li></ul></ul><ul><ul><ul><li>SIP for media configuration, event notification, instant multimedia messaging </li></ul></ul></ul><ul><ul><ul><li>SLP for service discovery </li></ul></ul></ul><ul><ul><li>integration of presence and user context </li></ul></ul><ul><ul><ul><li>standardization in the IETF (RPID) </li></ul></ul></ul><ul><ul><li>location-based services </li></ul></ul><ul><ul><ul><li>user context </li></ul></ul></ul><ul><ul><ul><li>user authorization </li></ul></ul></ul><ul><ul><ul><li>service location </li></ul></ul></ul>
  15. 15. Mobility in Pervasive Pixels <ul><li>Terminal mobility </li></ul><ul><ul><li>application-layer mobility complements L3 mobility </li></ul></ul><ul><li>Session mobility </li></ul><ul><ul><li>move active sessions to devices found in the environment </li></ul></ul><ul><ul><li> service discovery </li></ul></ul><ul><li>Service mobility </li></ul><ul><ul><li>move configuration to new devices </li></ul></ul><ul><li>Personal mobility </li></ul><ul><ul><li>one user, many devices </li></ul></ul>
  16. 16. Location-based services <ul><li>Traditionally, focus on geospatial location (e.g., GPS) </li></ul><ul><li>But other aspects as important: </li></ul><ul><ul><li>civil location (often more intuitive) </li></ul></ul><ul><ul><li>type of place (home vs. office; outdoors vs. theatre) </li></ul></ul><ul><ul><li>behavioral: distraction, privacy, appropriateness </li></ul></ul><ul><li>Experimenting with low-complexity location mechanisms: </li></ul><ul><ul><li>IR/RF active badges with low installation cost (Ivistar) </li></ul></ul><ul><ul><li>BlueTooth location beacons </li></ul></ul><ul><ul><li>LAN backtracking and DHCP </li></ul></ul><ul><ul><li>swipe cards and i-buttons </li></ul></ul>DHCP server 458/17  Rm. 815 458/18  Rm. 816 DHCP answer: sta=DC loc=Rm815 lat=38.89868 long=77.03723 8:0:20:ab:d5:d CDP + SNMP 8:0:20:ab:d5:d  458/17
  17. 17. Some initial lessons learned <ul><li>Usage: remote presence from UKy during sabbatical </li></ul><ul><ul><li>research group meetings </li></ul></ul><ul><ul><li>departmental site visit </li></ul></ul><ul><ul><li>thesis proposals and defenses </li></ul></ul><ul><li>Perception: “Multimedia collaboration is a mature field” </li></ul><ul><li>Reality: It doesn’t work much better than in 1992 </li></ul><ul><ul><li>still fails in hard-to-diagnose ways </li></ul></ul><ul><ul><li>quality better, but echo, feedback and level issues remain </li></ul></ul><ul><li>Integration between synchronous and asynchronous collaboration </li></ul><ul><ul><li>integrating documents, minutes, … </li></ul></ul><ul><li>Transition from call-focused to presence-focused </li></ul><ul><ul><li>much larger use of asynchronous collaboration (email, bulletin boards, …) </li></ul></ul><ul><li>Working with start-up company: new IP-based departmental communication system to replace PBX </li></ul>
  18. 18. Columbia SIP servers (CINEMA) Internal Telephone Extn: 7040 SIP/PSTN Gateway Department PBX Web based configuration Telephone switch SQL database sipd: Proxy, redirect, registrar server Extn: 7134 [email_address] NetMeeting H.323 rtspd: media server sipum: Unified messaging Quicktime RTSP clients RTSP Extn: 7136 713x Single machine SNMP (Network Management) sipconf: Conference server siph323: SIP-H.323 translator Local/long distance 1-212-5551212 Web server
  19. 19. Larger lessons for multimedia systems research <ul><li>Software tool support for multimedia communications lacking </li></ul><ul><ul><li>most are applications, not building blocks </li></ul></ul><ul><ul><li>cross-platform research media tools are getting very old and creaky (vic, rat, etc.) </li></ul></ul><ul><ul><li>multi-party support very weak (multicast never happened) </li></ul></ul><ul><li>Components designed to be operated by humans </li></ul><ul><ul><li>IP phones only have HTTP/HTML interface </li></ul></ul><ul><ul><li>video projectors just proprietary configuration </li></ul></ul><ul><li>Lots of components, but hard to evaluate in real use </li></ul><ul><ul><li>still mostly barely demo quality: audio delay, echo, random failures </li></ul></ul><ul><ul><li>people will fall back to good ol’ PSTN quickly </li></ul></ul>
  20. 20. Conclusion <ul><li>Pervasive Pixels = attempt to integrate multiple modalities into system, not just grouping of components </li></ul><ul><li>Evaluation in real usage, not just demos </li></ul><ul><li>Spread throughout the department, not just lab </li></ul>

×