RELIVING ON DEMAND: A TOTAL VIEWEREXPERIENCE           Vivek K. Singh1*, Jiebo Luo2, Dhiraj Joshi2,          Phoury Lei2, ...
Why do people take pictures? 1. Digital re-living 2. Sharing it with   family and friends
What’s available today?• Commercial Slideshows (Picasa, iPhoto, ACDsee):  • Focus on visual appearance only.  • Don’t unde...
Platforms   Desktop   Digital frame   HDTV   Kodak Gallery   Mobile   Kiosk
Preview • Re-living of events in user’s life, based on WHO,   WHERE, and WHEN .
Outline • Preview • Design principles • System design • Under the hood (sneak peek) • Evaluations
Design principles 1. User controllable:    • Responsive to user demand (overcoming intent gap) 2. Semantically drivable:  ...
Retrieval vs. Browsing vs. Reliving• Media by itself is uninteresting unless it performs a  function (e.g. reliving, shari...
System overview
System overview: Approach
Media data structure                                                          Media                               URL     ...
Pre-processing                                       Media                                      Collection       Date and ...
Reordering of event list• Basic idea• Time• People• Location
Choosing layout • Default:i=     2          3   4   5
Choose transitions• If (criteria=time || criteria=loc)   • Slide In/Out• If (criteria=personi)   • Face2Face transition  T...
Choose song• If (criteria=time)   • Select seasonal songs (easily extensible to finer grain)• If (criteria=loc)   • Select...
Show images • In time order • Higher score => more display time • Auto-zoom-crop   • Find center to focus on   • Match the...
Logging user sessions     <Interaction>                      <Click>                                    <GlobalEventID>urn...
Evaluations• Experiments with 11 families• 35 user interaction sessions logged          Age of contributing photographers ...
Experiment 1: Comparison with commercially availableoptions
6.2 Experiment 2: Use of different features acrossdifferent user demographics         Females   1.14      1.49            ...
Future work • Choosing songs more generically/smartly • Choosing optimal spatio-temporal placement of   images in the slid...
Upcoming SlideShare
Loading in …5
×

Reliving on demand a total viewer experience

344
-1

Published on

Enabling
media reliving experiences that are aesthetically pleasing,
interactive, and semantically drivable as they center on people,
locations, time, and events discovered in a media collection.

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
344
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • People don’t want to see images, they want to re-live and share the experiences
  • Reliving on demand a total viewer experience

    1. 1. RELIVING ON DEMAND: A TOTAL VIEWEREXPERIENCE Vivek K. Singh1*, Jiebo Luo2, Dhiraj Joshi2, Phoury Lei2, Madirakshi Das2, Peter Stubler2 1 University of California, Irvine, 2 Kodak Research Laboratories, Rochester, NY, ACM International Conference on Multimedia – ACMM 20111 * Work was done when the author was interning at Kodak Research Laboratories, Eastman Kodak Company, Rochester, NY, USA.
    2. 2. Why do people take pictures? 1. Digital re-living 2. Sharing it with family and friends
    3. 3. What’s available today?• Commercial Slideshows (Picasa, iPhoto, ACDsee): • Focus on visual appearance only. • Don’t understand/utilize semantics (except “FaceMovie”)• Research efforts: Semantic analysis • No interaction • Interaction on demand• Allow different users to dynamically re-direct the flow of media reliving experience
    4. 4. Platforms Desktop Digital frame HDTV Kodak Gallery Mobile Kiosk
    5. 5. Preview • Re-living of events in user’s life, based on WHO, WHERE, and WHEN .
    6. 6. Outline • Preview • Design principles • System design • Under the hood (sneak peek) • Evaluations
    7. 7. Design principles 1. User controllable: • Responsive to user demand (overcoming intent gap) 2. Semantically drivable: • Events as organizing units • Who, when, where; what 3. Aesthetically pleasing: • Dynamic presentation • Multimodal (songs, images, videos)
    8. 8. Retrieval vs. Browsing vs. Reliving• Media by itself is uninteresting unless it performs a function (e.g. reliving, sharing) for the human user• Retrieval • Fetching data. Strong intent (e.g. search)• Browsing • Piecemeal reliving. Weak intent (e.g. youtube)• Reliving • Valuable middle ground. • Semantically re-direct the flow if desired.
    9. 9. System overview
    10. 10. System overview: Approach
    11. 11. Media data structure Media URL properties Type Height, width Aesthetic Aesthetic IVI properties location subjects dateTime Semantic properties Score Suitability properties
    12. 12. Pre-processing Media Collection Date and Time Aesthetics Value Face Detection Location Information Extraction Extraction Extraction Face Clustering Event Clustering Face Labeling Geographic Clustering Metadata Repository
    13. 13. Reordering of event list• Basic idea• Time• People• Location
    14. 14. Choosing layout • Default:i= 2 3 4 5
    15. 15. Choose transitions• If (criteria=time || criteria=loc) • Slide In/Out• If (criteria=personi) • Face2Face transition Transform(θ1, trans.X Transform(θ2, trans.X 1, 2, trans.Y 1, scale 1) trans.Y2, scale 2)
    16. 16. Choose song• If (criteria=time) • Select seasonal songs (easily extensible to finer grain)• If (criteria=loc) • Select regional songs• If (criteria=personi) • Select age-based songs (easily extensible to gender)• Taken from a library of available songs
    17. 17. Show images • In time order • Higher score => more display time • Auto-zoom-crop • Find center to focus on • Match the aspect ratio required • Multiple Holes in transitions • Token passing amongst holes • Representative image as background
    18. 18. Logging user sessions <Interaction> <Click> <GlobalEventID>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</GlobalEventID> <SortedEventID>0</SortedEventID <TimeStamp>10:17:47 AM</TimeStamp> <Criteria_type>gps</Criteria_type> <Criteria_value>61.2175937710438 , -149.898739309764</Criteria_value> <HotSpotClick>False</HotSpotClick> </Click> <Snapshot> <Locations> <loc>-149.898739309764,61.2175937710438</loc> <loc>-73.508556462585,40.5956603174603</loc> <loc>102.757525301205,25.1018832329317</loc> <loc>104.195397,35.86166</loc> <loc>6.09306585111111,52.7236709366667</loc> </Locations> <People> <peo>Jiebo</peo> <peo>Joyce</peo> <peo>Xinping</peo> <peo></peo> <peo></peo> </People> <SortedEvents> <eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve> <eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve> <eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve> <eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve> <eve>urn:guid:f1337996-3c28-4345-b4fb-c4f1b788fc05</eve> <eve></eve> </SortedEvents> <PicsShown> <pic>c:datajiebocvpr2008103_5972.jpg</pic> <pic>c:datajiebocvpr2008103_5973.jpg</pic> <pic>c:datajiebolijiang-shangrila-day2108_0043.jpg</pic> <pic>c:datajiebolijiang-shangrila-day2108_0044.jpg</pic> </PicsShown> </Snapshot> </Interaction>
    19. 19. Evaluations• Experiments with 11 families• 35 user interaction sessions logged Age of contributing photographers 23 to 56 No. of images/ videos in the collection 2,091 to 10,522 No. of calendar years in time span 3 to 10 No. of tagged people in the collection 26 to 137 No. of places in the collection 19 to 45• Roles • 1st person (owner) • 2nd person (immediate family) • 3rd person (friends, cousins )
    20. 20. Experiment 1: Comparison with commercially availableoptions
    21. 21. 6.2 Experiment 2: Use of different features acrossdifferent user demographics Females 1.14 1.49 1.13 1.01 Males 1.41 1.25 2.08 1.43 Both 1.30 1.27 1.28 1.35 All 1st party 2nd party 3rd party Active Vs Passive? Clicks per axis Stickiness :Time spent after clicks
    22. 22. Future work • Choosing songs more generically/smartly • Choosing optimal spatio-temporal placement of images in the slide show • Choosing layout • Choosing transition time? • Supporting multiple axes simultaneously • Previews
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×