SlideShare a Scribd company logo
1
Smart Photo Selection:
Interpret Gaze as Personal Interest
Tina Walber1
, Ansgar Scherp2,3
, Steffen Staab1
1 Institute WeST, University of Koblenz, Germany
2 Kiel University, Germany
3 Leibniz Information Center for Economics, Kiel, Germany
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 2
Managment of Digital Photos
● Its a mess!
● We take a lot of photos
● Manually selecting photos is cumbersome
● Like to have photo selections for
– Sharing photos online
– Creating photo products like photo books
– Creating presentations
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 3
State of the Art: Automatic
Creation of Photo Selections
● Content-based approaches
– Analysis of low-level features
● Context-based approaches
– Analysis of context information
● What about individual aestetics, personal
preferences, user interests, ….?
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 4
Interpret Gaze as Personal Interest
● Gaze delivers information on user's interest
● Useful for creating individual photo selections?
● Principal approach
– Merely observe what users are doing anyway
– Do not ask to perfom additional tasks
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 5
● Starting from $99
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 6
Research Questions
1. Is there a need for individual photo selections?
2. Does a gaze-based selection outperform
selections based on content and context analysis
when comparing to those created manually?
3. Does the personal interest in a viewed photo set
have an impact on the obtained selection results?
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 7
Experiment Setup
Photo Viewing
Task:
„get an overview“
Step 1
Recording of the
eye tracking data
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 8
Photo Viewing
32 pages with 9 photos each
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 9
Experiment Setup
Photo Viewing
Task:
„get an overview“
Step 1
Photo Selection
Task: „select photos
for your private photo
collection“
Step 2
Recording of the
eye tracking data
Creation of
Ground Truth
Sm
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 10
Manual Selection
Creator:LibreOffice 3.5
LanguageLevel:2
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 11
Collection CA Collection CB
162 photos 126 photos
Experiment Data Set C = CA CB
∩
Two Data Sets and
Two User Groups
Institute A Institute B
Home collectionHome collection
Foreign collection
● Taken during social events of the research institutes
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 12
Participants
● 33 participants (12 of them female)
● 21 associated to Institute A, 12 to Institute B
● Aged between 25 and 62 (Ø 33.5 ± 9.57)
● 20 graduate students, 4 postdocs,
9 other professions
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 13
Overview Analysis and Evaluation
Se
Collection C
Gaze
Based
Selection
Calculation
of Precision
P
Manual
Selection
Content and
Context Based
Selection
Sb+e
Sb
Ground Truth
Sm
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 14
Baseline Measures
# Name Description
1 concentration
Time
Photo was taken with other photos in a
short period of time
2 sharpness Sharpness score from related work
3 numberOfFaces Number of faces
4 faceGaussian Size and position of faces
5 personsPopula
rity
Popularity of the depicted persons
6 faceArea Areas in pixels covered by faces
Selection of photos based on:
Calculated for each photo
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 15
Eye Tracking Data
●
Fixations and saccades
● Analysed gaze data with eye tracking measures
Creator:LibreOffice 3.5
LanguageLevel:2
● Viewing duration / page:
M = 12.6 s
● Number of fixations /
photo: M = 3.25
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 16
Eye Tracking Measures
# Name Description
7 fixated Was the photo was fixated?
Creator:LibreOffice 3.5
LanguageLevel:2
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 17
Eye Tracking Measures
# Name Description
7 fixated Was the photo was fixated?
8 fixationCount Counts the number of fixations
Creator:LibreOffice 3.5
LanguageLevel:2
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 18
Eye Tracking Measures
# Name Description
7 fixated Was the photo was fixated?
8 fixationCount Counts the number of fixations
9 fixationDuration Sum of duration of all fixations
Creator:LibreOffice 3.5
LanguageLevel:2
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 19
Eye Tracking Measures
# Name Description
7 fixated Was the photo was fixated?
8 fixationCount Counts the number of fixations
9 fixationDuration Sum of duration of all fixations
10 firstFixationDuration Duration of the first fixation
11 lastFixationDuration Duration of the last fixation
12 avgFixationDuration Average fixation duration
13 maxVisitDuration Maximum visit length
14 meanVisitDuration Average visit length
15 visitCount Number of visits
16 saccLength Mean length of the saccades
17 pupilMax Maximum pupil diameter
18 pupilMaxChange Maximum pupil diameter change
19 pupilAvg Average pupil diameter
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 20
Combination of Measures
● Using a model learned from logistic regression
● Assigns each image a probability of being
selected
● 30 random splits for training and test data
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 21
1. Is there a need for individual
photo selections?
1 21 41 61 81 101121141161181201221241261281
0
5
10
15
20
25
30
Photo with the highest number of selections
Photos with no selections
Photos in data set C
10 40 70 100 130 160 190 220 250 280
SelectionFrequency
● Manually created photo selections are diverse
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 22
2. Evaluation of the Photo Selections
PrecisionP
Sb Sb+e Se
*
*
Random
Selection
P = 0.428P = 0.365 P = 0.426
● Improvement of 17% over baseline
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 23
3. Impact of personal interest?
PrecisionP
Results for Sb+e
Foreign Collection Home Collection
P = 0.446P = 0.404
*
Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 24
Conclusion
● Photo selection behavior is individual
● Gaze helps capture personal preferences
● Results are better for photos with personal interest
● Might work even better for real personal photos
● Potential application in photo book authoring
Thank you for your attention!

More Related Content

More from ZBW - Leibniz-Informationszentrum Wirtschaft

„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...
„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...
„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...ZBW - Leibniz-Informationszentrum Wirtschaft
 
Open Assessment and Open Citation Analysis - Experiences with the Journal "E...
Open Assessment  and Open Citation Analysis - Experiences with the Journal "E...Open Assessment  and Open Citation Analysis - Experiences with the Journal "E...
Open Assessment and Open Citation Analysis - Experiences with the Journal "E...
ZBW - Leibniz-Informationszentrum Wirtschaft
 
Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...
Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...
Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...
ZBW - Leibniz-Informationszentrum Wirtschaft
 

More from ZBW - Leibniz-Informationszentrum Wirtschaft (20)

Literaturverwaltung & Bibliotheken - Das zentrale Portal für Informationen ...
Literaturverwaltung & Bibliotheken  - Das zentrale Portal  für Informationen ...Literaturverwaltung & Bibliotheken  - Das zentrale Portal  für Informationen ...
Literaturverwaltung & Bibliotheken - Das zentrale Portal für Informationen ...
 
Bibliotheken im Spannungsfeld zwischen Nutzungsmessung und Datenschutz
Bibliotheken im Spannungsfeld zwischen Nutzungsmessung und DatenschutzBibliotheken im Spannungsfeld zwischen Nutzungsmessung und Datenschutz
Bibliotheken im Spannungsfeld zwischen Nutzungsmessung und Datenschutz
 
EconDesk - Die Online-Auskunft für die Wirtschaftswissenschaften
EconDesk -   Die Online-Auskunft für die WirtschaftswissenschaftenEconDesk -   Die Online-Auskunft für die Wirtschaftswissenschaften
EconDesk - Die Online-Auskunft für die Wirtschaftswissenschaften
 
„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...
„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...
„Gemeinsam sind wir stärker“… - Erzielte Synergien beim Aufbau eines kooperat...
 
Leibniz -Bibliotheksverbund Forschungsinformation - Entwicklung und Anwendun...
Leibniz -Bibliotheksverbund Forschungsinformation  - Entwicklung und Anwendun...Leibniz -Bibliotheksverbund Forschungsinformation  - Entwicklung und Anwendun...
Leibniz -Bibliotheksverbund Forschungsinformation - Entwicklung und Anwendun...
 
EconBiz mobil: Fachrecherchen mit einer App
EconBiz mobil: Fachrecherchen mit einer AppEconBiz mobil: Fachrecherchen mit einer App
EconBiz mobil: Fachrecherchen mit einer App
 
Von der Sammlung gedruckter Statistiken zum Datenrepository
Von der Sammlung gedruckter Statistiken zum DatenrepositoryVon der Sammlung gedruckter Statistiken zum Datenrepository
Von der Sammlung gedruckter Statistiken zum Datenrepository
 
Research Papers in Economics: RePEc - ein fachliches Repository?
Research Papers in Economics: RePEc - ein fachliches Repository?Research Papers in Economics: RePEc - ein fachliches Repository?
Research Papers in Economics: RePEc - ein fachliches Repository?
 
Open Access-Publizieren in den Wirtschaftswissenschaften
Open Access-Publizieren in den WirtschaftswissenschaftenOpen Access-Publizieren in den Wirtschaftswissenschaften
Open Access-Publizieren in den Wirtschaftswissenschaften
 
Wie verbreitet sich die digitale Publikation im Netz?
Wie verbreitet sich die digitale Publikation im Netz?Wie verbreitet sich die digitale Publikation im Netz?
Wie verbreitet sich die digitale Publikation im Netz?
 
EconStor - ein vernetztes Fach-Repositorium
EconStor - ein vernetztes Fach-RepositoriumEconStor - ein vernetztes Fach-Repositorium
EconStor - ein vernetztes Fach-Repositorium
 
ZBW / Inhalte der Werbe-Displays, Hamburg
ZBW / Inhalte der Werbe-Displays, HamburgZBW / Inhalte der Werbe-Displays, Hamburg
ZBW / Inhalte der Werbe-Displays, Hamburg
 
Recherchieren wie die Profis - Tipps von der ZBW - Teil 13: Open Access - Eig...
Recherchieren wie die Profis - Tipps von der ZBW - Teil 13: Open Access - Eig...Recherchieren wie die Profis - Tipps von der ZBW - Teil 13: Open Access - Eig...
Recherchieren wie die Profis - Tipps von der ZBW - Teil 13: Open Access - Eig...
 
Recherchieren wie die Profis - Tipps von der ZBW - Teil 12: Was kann Google S...
Recherchieren wie die Profis - Tipps von der ZBW - Teil 12: Was kann Google S...Recherchieren wie die Profis - Tipps von der ZBW - Teil 12: Was kann Google S...
Recherchieren wie die Profis - Tipps von der ZBW - Teil 12: Was kann Google S...
 
Econbiz Einführung
Econbiz EinführungEconbiz Einführung
Econbiz Einführung
 
Fachliche Vernetzung von Repositories auf europäischer Ebene - das Beispiel N...
Fachliche Vernetzung von Repositories auf europäischer Ebene - das Beispiel N...Fachliche Vernetzung von Repositories auf europäischer Ebene - das Beispiel N...
Fachliche Vernetzung von Repositories auf europäischer Ebene - das Beispiel N...
 
Open Assessment and Open Citation Analysis - Experiences with the Journal "E...
Open Assessment  and Open Citation Analysis - Experiences with the Journal "E...Open Assessment  and Open Citation Analysis - Experiences with the Journal "E...
Open Assessment and Open Citation Analysis - Experiences with the Journal "E...
 
Das Weiterbildungsprogramm Bibliotheksmanagement der FU Berlin, von Olaf Siegert
Das Weiterbildungsprogramm Bibliotheksmanagement der FU Berlin, von Olaf SiegertDas Weiterbildungsprogramm Bibliotheksmanagement der FU Berlin, von Olaf Siegert
Das Weiterbildungsprogramm Bibliotheksmanagement der FU Berlin, von Olaf Siegert
 
Forschungsdaten im Kontext von Open-Access-Publikationsdiensten, von Olaf Sie...
Forschungsdaten im Kontext von Open-Access-Publikationsdiensten, von Olaf Sie...Forschungsdaten im Kontext von Open-Access-Publikationsdiensten, von Olaf Sie...
Forschungsdaten im Kontext von Open-Access-Publikationsdiensten, von Olaf Sie...
 
Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...
Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...
Open Access and Open Assessment: The Future of Peer Reviewed Journals? Experi...
 

Recently uploaded

如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
yqqaatn0
 
Cytokines and their role in immune regulation.pptx
Cytokines and their role in immune regulation.pptxCytokines and their role in immune regulation.pptx
Cytokines and their role in immune regulation.pptx
Hitesh Sikarwar
 
Phenomics assisted breeding in crop improvement
Phenomics assisted breeding in crop improvementPhenomics assisted breeding in crop improvement
Phenomics assisted breeding in crop improvement
IshaGoswami9
 
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
University of Maribor
 
8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf
by6843629
 
bordetella pertussis.................................ppt
bordetella pertussis.................................pptbordetella pertussis.................................ppt
bordetella pertussis.................................ppt
kejapriya1
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
University of Hertfordshire
 
Nucleophilic Addition of carbonyl compounds.pptx
Nucleophilic Addition of carbonyl  compounds.pptxNucleophilic Addition of carbonyl  compounds.pptx
Nucleophilic Addition of carbonyl compounds.pptx
SSR02
 
Oedema_types_causes_pathophysiology.pptx
Oedema_types_causes_pathophysiology.pptxOedema_types_causes_pathophysiology.pptx
Oedema_types_causes_pathophysiology.pptx
muralinath2
 
mô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốt
mô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốtmô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốt
mô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốt
HongcNguyn6
 
Deep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless ReproducibilityDeep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless Reproducibility
University of Rennes, INSA Rennes, Inria/IRISA, CNRS
 
Chapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisisChapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisis
tonzsalvador2222
 
原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样
原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样
原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样
yqqaatn0
 
Eukaryotic Transcription Presentation.pptx
Eukaryotic Transcription Presentation.pptxEukaryotic Transcription Presentation.pptx
Eukaryotic Transcription Presentation.pptx
RitabrataSarkar3
 
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...
Travis Hills MN
 
Topic: SICKLE CELL DISEASE IN CHILDREN-3.pdf
Topic: SICKLE CELL DISEASE IN CHILDREN-3.pdfTopic: SICKLE CELL DISEASE IN CHILDREN-3.pdf
Topic: SICKLE CELL DISEASE IN CHILDREN-3.pdf
TinyAnderson
 
20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx
Sharon Liu
 
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptx
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptxThe use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptx
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptx
MAGOTI ERNEST
 
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptx
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptxANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptx
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptx
RASHMI M G
 
SAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdfSAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdf
KrushnaDarade1
 

Recently uploaded (20)

如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
如何办理(uvic毕业证书)维多利亚大学毕业证本科学位证书原版一模一样
 
Cytokines and their role in immune regulation.pptx
Cytokines and their role in immune regulation.pptxCytokines and their role in immune regulation.pptx
Cytokines and their role in immune regulation.pptx
 
Phenomics assisted breeding in crop improvement
Phenomics assisted breeding in crop improvementPhenomics assisted breeding in crop improvement
Phenomics assisted breeding in crop improvement
 
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...
 
8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf8.Isolation of pure cultures and preservation of cultures.pdf
8.Isolation of pure cultures and preservation of cultures.pdf
 
bordetella pertussis.................................ppt
bordetella pertussis.................................pptbordetella pertussis.................................ppt
bordetella pertussis.................................ppt
 
Applied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdfApplied Science: Thermodynamics, Laws & Methodology.pdf
Applied Science: Thermodynamics, Laws & Methodology.pdf
 
Nucleophilic Addition of carbonyl compounds.pptx
Nucleophilic Addition of carbonyl  compounds.pptxNucleophilic Addition of carbonyl  compounds.pptx
Nucleophilic Addition of carbonyl compounds.pptx
 
Oedema_types_causes_pathophysiology.pptx
Oedema_types_causes_pathophysiology.pptxOedema_types_causes_pathophysiology.pptx
Oedema_types_causes_pathophysiology.pptx
 
mô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốt
mô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốtmô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốt
mô tả các thí nghiệm về đánh giá tác động dòng khí hóa sau đốt
 
Deep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless ReproducibilityDeep Software Variability and Frictionless Reproducibility
Deep Software Variability and Frictionless Reproducibility
 
Chapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisisChapter 12 - climate change and the energy crisis
Chapter 12 - climate change and the energy crisis
 
原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样
原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样
原版制作(carleton毕业证书)卡尔顿大学毕业证硕士文凭原版一模一样
 
Eukaryotic Transcription Presentation.pptx
Eukaryotic Transcription Presentation.pptxEukaryotic Transcription Presentation.pptx
Eukaryotic Transcription Presentation.pptx
 
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...
 
Topic: SICKLE CELL DISEASE IN CHILDREN-3.pdf
Topic: SICKLE CELL DISEASE IN CHILDREN-3.pdfTopic: SICKLE CELL DISEASE IN CHILDREN-3.pdf
Topic: SICKLE CELL DISEASE IN CHILDREN-3.pdf
 
20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx20240520 Planning a Circuit Simulator in JavaScript.pptx
20240520 Planning a Circuit Simulator in JavaScript.pptx
 
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptx
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptxThe use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptx
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptx
 
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptx
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptxANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptx
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptx
 
SAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdfSAR of Medicinal Chemistry 1st by dk.pdf
SAR of Medicinal Chemistry 1st by dk.pdf
 

Smart Photo Selection: Interpret Gaze as Personal Interest

  • 1. 1 Smart Photo Selection: Interpret Gaze as Personal Interest Tina Walber1 , Ansgar Scherp2,3 , Steffen Staab1 1 Institute WeST, University of Koblenz, Germany 2 Kiel University, Germany 3 Leibniz Information Center for Economics, Kiel, Germany
  • 2. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 2 Managment of Digital Photos ● Its a mess! ● We take a lot of photos ● Manually selecting photos is cumbersome ● Like to have photo selections for – Sharing photos online – Creating photo products like photo books – Creating presentations
  • 3. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 3 State of the Art: Automatic Creation of Photo Selections ● Content-based approaches – Analysis of low-level features ● Context-based approaches – Analysis of context information ● What about individual aestetics, personal preferences, user interests, ….?
  • 4. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 4 Interpret Gaze as Personal Interest ● Gaze delivers information on user's interest ● Useful for creating individual photo selections? ● Principal approach – Merely observe what users are doing anyway – Do not ask to perfom additional tasks
  • 5. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 5 ● Starting from $99
  • 6. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 6 Research Questions 1. Is there a need for individual photo selections? 2. Does a gaze-based selection outperform selections based on content and context analysis when comparing to those created manually? 3. Does the personal interest in a viewed photo set have an impact on the obtained selection results?
  • 7. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 7 Experiment Setup Photo Viewing Task: „get an overview“ Step 1 Recording of the eye tracking data
  • 8. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 8 Photo Viewing 32 pages with 9 photos each
  • 9. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 9 Experiment Setup Photo Viewing Task: „get an overview“ Step 1 Photo Selection Task: „select photos for your private photo collection“ Step 2 Recording of the eye tracking data Creation of Ground Truth Sm
  • 10. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 10 Manual Selection Creator:LibreOffice 3.5 LanguageLevel:2
  • 11. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 11 Collection CA Collection CB 162 photos 126 photos Experiment Data Set C = CA CB ∩ Two Data Sets and Two User Groups Institute A Institute B Home collectionHome collection Foreign collection ● Taken during social events of the research institutes
  • 12. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 12 Participants ● 33 participants (12 of them female) ● 21 associated to Institute A, 12 to Institute B ● Aged between 25 and 62 (Ø 33.5 ± 9.57) ● 20 graduate students, 4 postdocs, 9 other professions
  • 13. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 13 Overview Analysis and Evaluation Se Collection C Gaze Based Selection Calculation of Precision P Manual Selection Content and Context Based Selection Sb+e Sb Ground Truth Sm
  • 14. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 14 Baseline Measures # Name Description 1 concentration Time Photo was taken with other photos in a short period of time 2 sharpness Sharpness score from related work 3 numberOfFaces Number of faces 4 faceGaussian Size and position of faces 5 personsPopula rity Popularity of the depicted persons 6 faceArea Areas in pixels covered by faces Selection of photos based on: Calculated for each photo
  • 15. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 15 Eye Tracking Data ● Fixations and saccades ● Analysed gaze data with eye tracking measures Creator:LibreOffice 3.5 LanguageLevel:2 ● Viewing duration / page: M = 12.6 s ● Number of fixations / photo: M = 3.25
  • 16. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 16 Eye Tracking Measures # Name Description 7 fixated Was the photo was fixated? Creator:LibreOffice 3.5 LanguageLevel:2
  • 17. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 17 Eye Tracking Measures # Name Description 7 fixated Was the photo was fixated? 8 fixationCount Counts the number of fixations Creator:LibreOffice 3.5 LanguageLevel:2
  • 18. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 18 Eye Tracking Measures # Name Description 7 fixated Was the photo was fixated? 8 fixationCount Counts the number of fixations 9 fixationDuration Sum of duration of all fixations Creator:LibreOffice 3.5 LanguageLevel:2
  • 19. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 19 Eye Tracking Measures # Name Description 7 fixated Was the photo was fixated? 8 fixationCount Counts the number of fixations 9 fixationDuration Sum of duration of all fixations 10 firstFixationDuration Duration of the first fixation 11 lastFixationDuration Duration of the last fixation 12 avgFixationDuration Average fixation duration 13 maxVisitDuration Maximum visit length 14 meanVisitDuration Average visit length 15 visitCount Number of visits 16 saccLength Mean length of the saccades 17 pupilMax Maximum pupil diameter 18 pupilMaxChange Maximum pupil diameter change 19 pupilAvg Average pupil diameter
  • 20. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 20 Combination of Measures ● Using a model learned from logistic regression ● Assigns each image a probability of being selected ● 30 random splits for training and test data
  • 21. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 21 1. Is there a need for individual photo selections? 1 21 41 61 81 101121141161181201221241261281 0 5 10 15 20 25 30 Photo with the highest number of selections Photos with no selections Photos in data set C 10 40 70 100 130 160 190 220 250 280 SelectionFrequency ● Manually created photo selections are diverse
  • 22. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 22 2. Evaluation of the Photo Selections PrecisionP Sb Sb+e Se * * Random Selection P = 0.428P = 0.365 P = 0.426 ● Improvement of 17% over baseline
  • 23. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 23 3. Impact of personal interest? PrecisionP Results for Sb+e Foreign Collection Home Collection P = 0.446P = 0.404 *
  • 24. Walber, Scherp, Staab ● Smart Photo Selection: Interpret Gaze as Personal Interest 24 Conclusion ● Photo selection behavior is individual ● Gaze helps capture personal preferences ● Results are better for photos with personal interest ● Might work even better for real personal photos ● Potential application in photo book authoring Thank you for your attention!