© Ramesh Jain 1 Fall 2008
Upcoming SlideShare
Loading in...5
×
 

© Ramesh Jain 1 Fall 2008

on

  • 459 views

 

Statistics

Views

Total Views
459
Views on SlideShare
459
Embed Views
0

Actions

Likes
0
Downloads
27
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

© Ramesh Jain 1 Fall 2008 © Ramesh Jain 1 Fall 2008 Presentation Transcript

  • Fall 2008 CS 212 Multimedia Systems and Applications Instructor: Ramesh Jain
  • Projects Based on Research in Experiential Computing Laboratory
    • Based on the concept of EventWeb
    • Two majore research projects
      • Personal Media Management
      • Environment to Environment Connectivity (E2E)
  • Tim Berners-Lee thought:
    • Suppose all the information stored on computers everywhere were linked.
    • Suppose I could program my computer to create a space in which anything could be linked to anything.
    Others – including Bush -- had that idea earlier but the technology was not ready.
  • That resulted in the Web
    • DocumentWeb
      • Each node is a ‘Page’ or a document.
      • Pages are linked through explicit referential links
  • Continuing the Evolution of the Web
    • Consider a Web in which each node
      • Is an event
      • Has informational as well as experiential data
      • Is connected to other nodes using
        • Referential links
        • Structural links
        • Relational links
        • Causal links
      • Explicit links can be created by anybody
    • This EventWeb is connected to other Webs.
  • Different Facets of an Event
    • What
    • When
    • Where
    • Who
    • Why
    • How
  • Time 1- dimensional Space Events Happen
  • Time 1- dimensional Space EventWeb
  • Stage 1: Event Ingestion Atomic Event Entry Event Base
  • Stage 2: Event Clustering/Linking Event Grouping, Linking, Assimilation Event Base Atomic Event Entry
  • Stage 3: Event Presentation, Navigation Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry
  • Multimedia Storytelling
    • Collect information about events
      • All relevant events
      • Appropriate information
      • In right media
    • Stories are sequence of events.
      • Stories/Novels
      • Drama
      • Movies
    Present right event information using right media in right order.
  • Time 1- dimensional Space Multimedia Storytelling Text Video Photo
  • Experiential Media Management Environment
    • Event-based
    • Photo-taking is an event
    • Photos are taken of an event
    • Albums represents events and objects
    • Presentations are to share and communicate experiences
    First Photos then other media.
  • EMME Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry EXIF Features Tags/ Context
  • Modern Cameras
    • Are more than ‘Camera Obscura’.
    • Many sensors capture scene context and store it along with intensity values.
    • EXIF data is all metadata related to the Event .
    • And we throw away all this valuable information!!!
  • Examples: Photos are Assigned tags only based on EXIF
  • EMME Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry EXIF Features Tags/ Context Photo stream Segment. Event Ontology User Annot- ations
  • Photo Stream Segmentation
    • Definition: given a photo stream
    Segment Event 1 Event 2 Event 3
  • EMME Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry EXIF Features Tags/ Context Photo stream Segment. Event Ontology User Annot- ations Story Telling Search Explore
  • Using EMME
    • Searching for photo
      • I2R visit
    • Creating Albums:
      • Professional in Singapore
      • Tourism in Summer 2008
    • Telling stories
      • What did I do in Summer 2008
    • Scenario: In December 2008, I have 20,000 pictures taken in 2008. How do I (semi-automatically) select 25 to send to
      • My mother
      • The uncle that I hate
      • My personal friend
      • My professional friend
  • Sentient EventWeb
    • Use sensors to create events.
      • This class
    • Infrastructure to create and link all such events.
      • Index
      • Search
      • Present
    • Archive events and put them on the Web.
  • First Version: E2E
    • Person to Person
    • Device to Device
      • Telephone
      • Video Conferencing
    • Environment to Environment (E2E)
    Communication Modes:
  • Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry
  • Natural interaction Semantic interaction Seamless interaction Bi-directional connectivity Not depend on physical similarity Handle privacy Event-based architecture Scalable architecture Sensor abstraction Multimodal information No fixed application Live and archived modes E2E Communication
  • Environment
    • Fixed objects
    • Active Objects
    • Events and Activities
    • Sensor to capture Objects and Events
      • Cameras, microphones, pressure, RFID, …
    • Displays and Actuators
      • Projectors, Displays, Speakers, …
  • Key Idea
    • Put enough sensors and devices to make mediation ( sensors and devices ) disappear.
    • System detects events and objects and sends appropriate data streams and displays/renders incoming streams on appropriate devices
    • Bring back Person to Person communication
  • System Architecture ES 2 ES 1 ES 3 ES 4 ES 5 JSM 2 JSM 1
  • Node Architecture Sensors Data Acquisition & Analysis Situation Model Env. Model + Security settings Event Server Multimedia Database Presentation Event Base Distribution/ Networking
  • Event e.g. E1_Cam2, E1_Mic2 e.g. Lip-motion=true, Volume=High e.g. Env.1, Position: 102,145 Visitor Chair e.g. Telemedicine App., Consultation room e.g. Patient talking Data Acquisition & Analysis Env. Model Situation Model Event Detection Stages
  • Env. 1 Env. 4 JSM 2 JSM 1 Env. 5 Env. 3 Env. 2 Joint Situation Model
  • Telemedicine Application
  • Telemedicine Application
  • Many Potential Projects: Photos
    • Features to classify images:
      • Detect specific objects (sky, grass, mountains, …) and label images.
      • Face detection and recognition.
      • Person recognition – faces to other body information to person.
      • Some simple ‘event’ (like a talk, or a wedding) recognition from photographs.
    • Using calendar information to assign information to images.
    • Integrating Social Network Information In Personal Media Management
    • Mobile App for uploading Event information.  Can use Android APIs
    • Select 1 or a few images from a group to represent the group. This may be based on quality of images or content of images.
    • Prepare a collage of images to represent a group. Use some theme for selecting and displaying your images.
    • Voice annotation to photos and use of speech recognition to convert voice tags to text tags.
    • Summarize an Event from Photos from many sources. Many people shoot a photo for an event. How to create a good event representation from photos from multiple sources?
  • Many Potential Projects: E2E
    • Tracking people using multiple sensors.
    • Mobile E2E mode: switching from a mobile phone to a much richer environment.
    • Creating ‘virtual camera’ for heterogeneous camera connections.
    • Anywhere projector
    • IP Audio streaming: Send and receive audio by just specifying the IP address – without knowing the devices connected there.