• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
© Ramesh Jain 1 Fall 2008
 

© Ramesh Jain 1 Fall 2008

on

  • 340 views

 

Statistics

Views

Total Views
340
Views on SlideShare
340
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    © Ramesh Jain 1 Fall 2008 © Ramesh Jain 1 Fall 2008 Presentation Transcript

    • Fall 2008 CS 212 Multimedia Systems and Applications Instructor: Ramesh Jain
    • Projects Based on Research in Experiential Computing Laboratory
      • Based on the concept of EventWeb
      • Two majore research projects
        • Personal Media Management
        • Environment to Environment Connectivity (E2E)
    • Tim Berners-Lee thought:
      • Suppose all the information stored on computers everywhere were linked.
      • Suppose I could program my computer to create a space in which anything could be linked to anything.
      Others – including Bush -- had that idea earlier but the technology was not ready.
    • That resulted in the Web
      • DocumentWeb
        • Each node is a ‘Page’ or a document.
        • Pages are linked through explicit referential links
    • Continuing the Evolution of the Web
      • Consider a Web in which each node
        • Is an event
        • Has informational as well as experiential data
        • Is connected to other nodes using
          • Referential links
          • Structural links
          • Relational links
          • Causal links
        • Explicit links can be created by anybody
      • This EventWeb is connected to other Webs.
    • Different Facets of an Event
      • What
      • When
      • Where
      • Who
      • Why
      • How
    • Time 1- dimensional Space Events Happen
    • Time 1- dimensional Space EventWeb
    • Stage 1: Event Ingestion Atomic Event Entry Event Base
    • Stage 2: Event Clustering/Linking Event Grouping, Linking, Assimilation Event Base Atomic Event Entry
    • Stage 3: Event Presentation, Navigation Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry
    • Multimedia Storytelling
      • Collect information about events
        • All relevant events
        • Appropriate information
        • In right media
      • Stories are sequence of events.
        • Stories/Novels
        • Drama
        • Movies
      Present right event information using right media in right order.
    • Time 1- dimensional Space Multimedia Storytelling Text Video Photo
    • Experiential Media Management Environment
      • Event-based
      • Photo-taking is an event
      • Photos are taken of an event
      • Albums represents events and objects
      • Presentations are to share and communicate experiences
      First Photos then other media.
    • EMME Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry EXIF Features Tags/ Context
    • Modern Cameras
      • Are more than ‘Camera Obscura’.
      • Many sensors capture scene context and store it along with intensity values.
      • EXIF data is all metadata related to the Event .
      • And we throw away all this valuable information!!!
    • Examples: Photos are Assigned tags only based on EXIF
    • EMME Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry EXIF Features Tags/ Context Photo stream Segment. Event Ontology User Annot- ations
    • Photo Stream Segmentation
      • Definition: given a photo stream
      Segment Event 1 Event 2 Event 3
    • EMME Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry EXIF Features Tags/ Context Photo stream Segment. Event Ontology User Annot- ations Story Telling Search Explore
    • Using EMME
      • Searching for photo
        • I2R visit
      • Creating Albums:
        • Professional in Singapore
        • Tourism in Summer 2008
      • Telling stories
        • What did I do in Summer 2008
      • Scenario: In December 2008, I have 20,000 pictures taken in 2008. How do I (semi-automatically) select 25 to send to
        • My mother
        • The uncle that I hate
        • My personal friend
        • My professional friend
    • Sentient EventWeb
      • Use sensors to create events.
        • This class
      • Infrastructure to create and link all such events.
        • Index
        • Search
        • Present
      • Archive events and put them on the Web.
    • First Version: E2E
      • Person to Person
      • Device to Device
        • Telephone
        • Video Conferencing
      • Environment to Environment (E2E)
      Communication Modes:
    • Event Cycle Event Base Event Presentation/ Navigation Event Grouping, Linking, Assimilation Atomic Event Entry
    • Natural interaction Semantic interaction Seamless interaction Bi-directional connectivity Not depend on physical similarity Handle privacy Event-based architecture Scalable architecture Sensor abstraction Multimodal information No fixed application Live and archived modes E2E Communication
    • Environment
      • Fixed objects
      • Active Objects
      • Events and Activities
      • Sensor to capture Objects and Events
        • Cameras, microphones, pressure, RFID, …
      • Displays and Actuators
        • Projectors, Displays, Speakers, …
    • Key Idea
      • Put enough sensors and devices to make mediation ( sensors and devices ) disappear.
      • System detects events and objects and sends appropriate data streams and displays/renders incoming streams on appropriate devices
      • Bring back Person to Person communication
    • System Architecture ES 2 ES 1 ES 3 ES 4 ES 5 JSM 2 JSM 1
    • Node Architecture Sensors Data Acquisition & Analysis Situation Model Env. Model + Security settings Event Server Multimedia Database Presentation Event Base Distribution/ Networking
    • Event e.g. E1_Cam2, E1_Mic2 e.g. Lip-motion=true, Volume=High e.g. Env.1, Position: 102,145 Visitor Chair e.g. Telemedicine App., Consultation room e.g. Patient talking Data Acquisition & Analysis Env. Model Situation Model Event Detection Stages
    • Env. 1 Env. 4 JSM 2 JSM 1 Env. 5 Env. 3 Env. 2 Joint Situation Model
    • Telemedicine Application
    • Telemedicine Application
    • Many Potential Projects: Photos
      • Features to classify images:
        • Detect specific objects (sky, grass, mountains, …) and label images.
        • Face detection and recognition.
        • Person recognition – faces to other body information to person.
        • Some simple ‘event’ (like a talk, or a wedding) recognition from photographs.
      • Using calendar information to assign information to images.
      • Integrating Social Network Information In Personal Media Management
      • Mobile App for uploading Event information.  Can use Android APIs
      • Select 1 or a few images from a group to represent the group. This may be based on quality of images or content of images.
      • Prepare a collage of images to represent a group. Use some theme for selecting and displaying your images.
      • Voice annotation to photos and use of speech recognition to convert voice tags to text tags.
      • Summarize an Event from Photos from many sources. Many people shoot a photo for an event. How to create a good event representation from photos from multiple sources?
    • Many Potential Projects: E2E
      • Tracking people using multiple sensors.
      • Mobile E2E mode: switching from a mobile phone to a much richer environment.
      • Creating ‘virtual camera’ for heterogeneous camera connections.
      • Anywhere projector
      • IP Audio streaming: Send and receive audio by just specifying the IP address – without knowing the devices connected there.