Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
SelPh
Progressive Learning and Support of
Manual Photo Color Enhancement
Yuki Koyama, Daisuke Sakamoto, Takeo Igarashi
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Our Approach
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Related Work
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
A Prototype System
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
1. Visualization
2. Optimization
3. Auto-Enhancement
4. Variable Confidence
5. Reference Photos
Self-Reinforcement Algorithm
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
User Study
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Results
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016...
Conclusion
■ Self-reinforcing system for manual photo enhancement
- Progressively and implicitly learning the user’s preference
■ A p...
■ Self-reinforcing system for manual photo enhancement
- Progressively and implicitly learning the user’s preference
■ A p...
■ Self-reinforcing system for manual photo enhancement
- Progressively and implicitly learning the user’s preference
■ A p...
■ Self-reinforcing system for manual photo enhancement
- Progressively and implicitly learning the user’s preference
■ A p...
Paper / Videos / Software / Source Codes
are available at
http://koyama.xyz/project/SelPh/
SelPh
Progressive Learning and Support of
Manual Photo Color Enhancement
Yuki Koyama, Daisuke Sakamoto, Takeo Igarashi
[CHI 2016] SelPh: Progressive Learning and Support of Manual Photo Color Enhancement
[CHI 2016] SelPh: Progressive Learning and Support of Manual Photo Color Enhancement
[CHI 2016] SelPh: Progressive Learning and Support of Manual Photo Color Enhancement
Upcoming SlideShare
Loading in …5
×

[CHI 2016] SelPh: Progressive Learning and Support of Manual Photo Color Enhancement

4,325 views

Published on

Project page: http://koyama.xyz/project/SelPh/

Yuki Koyama, Daisuke Sakamoto, and Takeo Igarashi. 2016. SelPh: Progressive Learning and Support of Manual Photo Color Enhancement. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), pp.2520--2532.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

[CHI 2016] SelPh: Progressive Learning and Support of Manual Photo Color Enhancement

  1. 1. SelPh Progressive Learning and Support of Manual Photo Color Enhancement Yuki Koyama, Daisuke Sakamoto, Takeo Igarashi
  2. 2. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Parameter 
 tweaking - Based on 
 the designer’s
 aesthetic
 preference 3 Photo Color Enhancement [Adobe Photoshop]
  3. 3. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Tedious 4
  4. 4. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 5 Enhancing Many Photos
  5. 5. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■Could be tedious ■Need manual enhancement of each photo 
 one by one, independently 6 Enhancing Many Photos
  6. 6. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Tedious 7Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 Solution?: Auto-Enhancement
  7. 7. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 8 Auto-Enhancement is NOT Perfect Manual edit (ground truth) Original Auto [Adobe Photoshop]
  8. 8. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 9 Auto-Enhancement is NOT Perfect Different Manual edit (ground truth) Original Auto [Adobe Photoshop]
  9. 9. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 10 Auto-Enhancement is NOT Perfect Different Manual edit (ground truth) Original Auto [Adobe Photoshop]
  10. 10. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 11 Auto-Enhancement is NOT Perfect One-by-one manual tweaking is necessary Manual edit (ground truth) Original Auto [Adobe Photoshop]
  11. 11. Our Approach
  12. 12. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 13 Our Approach Achieving 
 automatic enhancement Supporting 
 manual enhancement
  13. 13. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 14 Our Approach Achieving 
 automatic enhancement Supporting 
 manual enhancement ✗ ✓
  14. 14. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 15 Our Approach Achieving 
 automatic enhancement Supporting 
 manual enhancement New workflow concept: Self-reinforcing color enhancement ✗ ✓
  15. 15. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 16 Enhancement with a “Self-Reinforcing” System
  16. 16. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 17 Enhancement with a “Self-Reinforcing” System Target photos User’s edits
  17. 17. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 18 Enhancement with a “Self-Reinforcing” System Target photos User’s edits
  18. 18. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 19 Enhancement with a “Self-Reinforcing” System Target photos User’s edits
  19. 19. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 20 Enhancement with a “Self-Reinforcing” System Target photos User’s edits …
  20. 20. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 21 Enhancement with a “Self-Reinforcing” System Learn Target photos User’s edits Preference model …
  21. 21. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 22 Enhancement with a “Self-Reinforcing” System Learn Support Target photos User’s edits Preference model …
  22. 22. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 23 Enhancement with a “Self-Reinforcing” System Learn Support Update Learn Target photos User’s edits Preference model …
  23. 23. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 24 Enhancement with a “Self-Reinforcing” System Learn Support Update Learn Support Target photos User’s edits Preference model …
  24. 24. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 25 Enhancement with a “Self-Reinforcing” System Learn Support Update Learn Support Update Learn Target photos User’s edits Preference model …
  25. 25. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 26 Enhancement with a “Self-Reinforcing” System Learn Support Update Learn Support Update Learn Support Target photos User’s edits Preference model …
  26. 26. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 27 Enhancement with a “Self-Reinforcing” System Learn Support Update Learn Support Update Learn Support Update Learn Target photos User’s edits Preference model …
  27. 27. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 28 Enhancement with a “Self-Reinforcing” System … … Learn Support Update Learn Support Update Learn Support Update Learn Support Update Target photos User’s edits Preference model …
  28. 28. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Implicit - The user can work as usual - No explicit training phase ■ Progressive - The more photos are enhanced, the more useful the system becomes 29 “Self-Reinforcing” System
  29. 29. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Implicit - The user can work as usual - No explicit training phase ■ Progressive - The more photos are enhanced, the more useful the system becomes 30 “Self-Reinforcing” System
  30. 30. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Implicit - The user can work as usual - No explicit training phase ■ Progressive - The more photos are enhanced, the more useful the system becomes 31 “Self-Reinforcing” System
  31. 31. Related Work
  32. 32. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 33 UIs for Design Exploration Design Galleries [Marks+, SIGGRAPH97] ■ Design Galleries - Sampling-based selection ■ Our approach - Direct manipulation of each parameter using slider
  33. 33. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 34 UIs for Design Exploration Design Galleries [Marks+, SIGGRAPH97] ■ Design Galleries - Sampling-based selection ■ Our approach - Direct manipulation of each parameter using slider
  34. 34. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 35 UIs for Design Exploration Design Galleries [Marks+, SIGGRAPH97] ■ Design Galleries - Sampling-based selection ■ Our approach - Direct manipulation of each parameter using slider
  35. 35. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 36 UIs for Design Exploration ■ VisOpt Slider
 [Koyama+, UIST14] - Visualization - Optimization
  36. 36. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 37 UIs for Design Exploration ■ VisOpt Slider
 [Koyama+, UIST14] - Visualization - Optimization
  37. 37. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 38 UIs for Design Exploration VisOpt Slider [Koyama+, UIST14] Adopted into our prototype system
  38. 38. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 39 UIs for Design Exploration VisOpt Slider [Koyama+, UIST14] ■ [Koyama+, UIST14] - Training Data: Crowdsourcing - Interaction: Pre-computing ■ This work - Training Data: Editing history - Interaction: Progressive / Implicit Adopted into our prototype system
  39. 39. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 40 UIs for Design Exploration VisOpt Slider [Koyama+, UIST14] ■ [Koyama+, UIST14] - Training Data: Crowdsourcing - Interaction: Pre-computing ■ This work - Training Data: Editing history - Interaction: Progressive / Implicit Adopted into our prototype system
  40. 40. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 41 UIs for Design Exploration VisOpt Slider [Koyama+, UIST14] ■ [Koyama+, UIST14] - Training Data: Crowdsourcing - Interaction: Pre-computing ■ This work - Training Data: Editing history - Interaction: Progressive / Implicit Adopted into our prototype system
  41. 41. A Prototype System
  42. 42. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 SelPh A self-reinforcing system for photo enhancement 43 A Prototype System
  43. 43. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 1. Goodness Visualization 2. Interactive Optimization 3. Auto-Enhancement 4. Variable Confidence 5. Reference Photos 44 User Support Functions
  44. 44. 1. Visualization
  45. 45. 2. Optimization
  46. 46. 3. Auto-Enhancement
  47. 47. 4. Variable Confidence
  48. 48. 5. Reference Photos
  49. 49. Self-Reinforcement Algorithm
  50. 50. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 52 Interactive enhancement
  51. 51. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 53 Interactive enhancement Push the “Next” button
  52. 52. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 54 Interactive enhancement Push the “Next” button Update the distance metric d( , ) = ?
  53. 53. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 55 Interactive enhancement Push the “Next” button Update the preference model Update Update the distance metric d( , ) = ?
  54. 54. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 56 Interactive enhancement Push the “Next” button Show the next photo Update the preference model Update Update the distance metric d( , ) = ?
  55. 55. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 57 Interactive enhancement Push the “Next” button Show the next photo Update the preference model Update Update the distance metric d( , ) = ?
  56. 56. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 58 Interactive enhancement Push the “Next” button Show the next photo Update the preference model Update Update the distance metric d( , ) = ?
  57. 57. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 59 Distance Metric Learning d( , ) = ?
  58. 58. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ We adopt techniques by [Kapoor+, IJCV14] - Learning a personalized distance metric from editing history 60 Distance Metric Learning d( , ) = ? d( , ) d( , )is as equivalent to as possibleS.t.
  59. 59. User Study
  60. 60. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Qualitative evaluation of our approach / system - How photographers enhance photos - How they are satisfied 63 Purpose
  61. 61. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Participants: 8 photographers ■ Pre-task: Take 100 photos ■ Main-tasks: - Enhance the first 50 photos using { Baseline - Enhance the other 50 photos using { SelPh | Baseline } ■ Post-task: Interviews and questionnaires 64 Study Method
  62. 62. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Participants: 8 photographers ■ Pre-task: Take 100 photos ■ Main-tasks: - Enhance the first 50 photos using { Baseline - Enhance the other 50 photos using { SelPh | Baseline } ■ Post-task: Interviews and questionnaires 65 Study Method
  63. 63. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Participants: 8 photographers ■ Pre-task: Take 100 photos ■ Main-tasks: - Enhance the first 50 photos using { Baseline - Enhance the other 50 photos using { SelPh | Baseline } ■ Post-task: Interviews and questionnaires 66 Study Method
  64. 64. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Participants: 8 photographers ■ Pre-task: Take 100 photos ■ Main-tasks: - Enhance the first 50 photos using { Baseline | SelPh } - Enhance the other 50 photos using { SelPh | Baseline } ■ Post-task: Interviews and questionnaires 67 Study Method
  65. 65. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Participants: 8 photographers ■ Pre-task: Take 100 photos ■ Main-tasks: - Enhance the first 50 photos using { Baseline | SelPh } - Enhance the other 50 photos using { SelPh | Baseline } ■ Post-task: Interviews and questionnaires 68 Study Method Prepared by limiting the user support functions of SelPh
  66. 66. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ Participants: 8 photographers ■ Pre-task: Take 100 photos ■ Main-tasks: - Enhance the first 50 photos using { Baseline | SelPh } - Enhance the other 50 photos using { SelPh | Baseline } ■ Post-task: Interviews and questionnaires 69 Study Method Prepared by limiting the user support functions of SelPh
  67. 67. Results
  68. 68. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 71 Post-Task Questionnaires Q1# Q2# Q3# Q4# Q5# Q6# Q1# Q2# Q3# Q4# Q5# Q6# Q7# Q8# Q2# Q3# Q4# Q5# Q6# Q7# Q8# Q9# Q1 Visualization of goodness on sliders was useful compared to the absence of it. Q2 Interactive optimization of slider values was useful compared to the absence of it. Q5 Auto-enhancement in SelPh was more useful than that in commercial software. Q7 Reference photos in SelPh were more useful than those in Baseline. Q8 Confidence value was useful. Strongly disagree Strongly agree About user support functions — Positive
  69. 69. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 72 Post-Task QuestionnairesQ6# Q7# Q8# Q9# Q10# Q11# Q9 As the task proceeds, I felt that the system learns my preference or intent. Q10 The learning result reflected my preference or intent. Q11 It is preferable for the system to learn my preference or intent. Strongly disagree Strongly agree About overall approach — Positive
  70. 70. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ About Overall Experience - “The functions [in SelPh] […] evoke the feeling of collaborating with another me.” - Using the baseline system, “[I felt] lonely.” In contrast, “there is interaction with the [self-reinforcing] system,” thus “executing the task [with SelPh] was fun.” 73 Comments from Interviews (1/2)
  71. 71. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ About Confidence Value - “[According to the confidence value, I] decided to use or not to use the optimization and the auto-enhancement” - “I could trust the system more [by knowing the confidence]” - “It was an enjoyable experience to do the task [while knowing the confidence]” - “[I felt] humanity from the confidence” 74 Comments from Interviews (2/2)
  72. 72. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ About Confidence Value - “[According to the confidence value, I] decided to use or not to use the optimization and the auto-enhancement” - “I could trust the system more [by knowing the confidence]” - “It was an enjoyable experience to do the task [while knowing the confidence]” - “[I felt] humanity from the confidence” 75 Comments from Interviews (2/2)
  73. 73. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ About Confidence Value - “[According to the confidence value, I] decided to use or not to use the optimization and the auto-enhancement” - “I could trust the system more [by knowing the confidence]” - “It was an enjoyable experience to do the task [while knowing the confidence]” - “[I felt] humanity from the confidence” 76 Comments from Interviews (2/2)
  74. 74. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ About Confidence Value - “[According to the confidence value, I] decided to use or not to use the optimization and the auto-enhancement” - “I could trust the system more [by knowing the confidence]” - “It was an enjoyable experience to do the task [while knowing the confidence]” - “[I felt] humanity from the confidence” 77 Comments from Interviews (2/2)
  75. 75. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 78 Quantitative Results Marginally faster [p < .10] Significantly smaller [p < .05]
  76. 76. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 79 Quantitative Results Marginally faster [p < .10] Significantly smaller [p < .05]
  77. 77. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 80 Quantitative Results Marginally faster [p < .10] Significantly smaller [p < .05]
  78. 78. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ All the participants were satisfied with self-reinforcing color enhancement ■ The inclusion of the confidence value makes SelPh more trustworthy and enjoyable to use 81 Summary & Lessons Learned
  79. 79. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ All the participants were satisfied with self-reinforcing color enhancement ■ The inclusion of the confidence value makes SelPh more trustworthy and enjoyable to use 82 Summary & Lessons Learned
  80. 80. Y. Koyama, D. Sakamoto, T. Igarashi | SelPh: Progressive Learning and Support of Manual Photo Color Enhancement | CHI 2016 ■ All the participants were satisfied with self-reinforcing color enhancement ■ The inclusion of the confidence value makes SelPh more trustworthy and enjoyable to use 83 Summary & Lessons Learned
  81. 81. Conclusion
  82. 82. ■ Self-reinforcing system for manual photo enhancement - Progressively and implicitly learning the user’s preference ■ A prototype system: SelPh - Five user support functions (e.g., variable confidence) ■ Qualitative user study - Photographers prefer the proposed workflow
  83. 83. ■ Self-reinforcing system for manual photo enhancement - Progressively and implicitly learning the user’s preference ■ A prototype system: SelPh - Five user support functions (e.g., variable confidence) ■ Qualitative user study - Photographers prefer the proposed workflow
  84. 84. ■ Self-reinforcing system for manual photo enhancement - Progressively and implicitly learning the user’s preference ■ A prototype system: SelPh - Five user support functions (e.g., variable confidence) ■ Qualitative user study - Photographers prefer the proposed workflow
  85. 85. ■ Self-reinforcing system for manual photo enhancement - Progressively and implicitly learning the user’s preference ■ A prototype system: SelPh - Five user support functions (e.g., variable confidence) ■ Qualitative user study - Photographers prefer the proposed workflow
  86. 86. Paper / Videos / Software / Source Codes are available at http://koyama.xyz/project/SelPh/
  87. 87. SelPh Progressive Learning and Support of Manual Photo Color Enhancement Yuki Koyama, Daisuke Sakamoto, Takeo Igarashi

×