Personal image retargeting

1. Green dominant
Red dominant

2. Image Retargeting with Personal
Saliency Map
Huan-Cheng Hsu

3. Introduction
• Period from 4:3 to 16:9

4. Introduction
• Various display media
Android iOS

5. Image Retargeting
Resize
Retarget

6. Image Retargeting
• Fit images to various display media
4:3
16:9 1:1

7. Saliency Map
• Traditional saliency map based on human
visual system performed well on many cases.

8. Retargeting Flow Chart
• Seam Carving
Looking for the most unimportant seam
Saliency Map

9. Personal Saliency Map
• However, everyone has his personal taste, a
fixed saliency map cannot meet everyone’s
requirement.
Great! This map
knows me!

10. Personal Saliency Map

11. Seam Carving with Personal Saliency
Map
Personal Saliency Map
??
Remove regions tester
did not like.

12. Personalization
• Question 1
– Which one of these two images makes you feel
beautiful?
• Question 2
– Reasons that why image you choose is beautiful?

13. Questionnaire
• For question one, without loss of generality,
we assume 50% people like maple leaves, 50%
people like blue rose

14. Questionnaire
Multiple
Objects
leaves
Red
Single Object
Rose
Blue

15. Observations
• It is hard to make all people choose the same
image that makes them feel beautiful
• Although people who select same image,
different reasons they might have
• Although people who have the same reason,
different degree might appear

16. Degree of Features
Red
Leaves
Multiple Objects
Location
Clearness
…
…
Gradient
0.5
0.2
0.3
0.2
0.1
0.7

17. Personalization
• Regression
Label
Value of Feature

18. Experiment Setting
• Setting of experiment:
– Use Color(RGB) as features
– Pictures with red, green and blue schemes
– Testers are asked to divide pictures into two group,
group of like and dislike , in order to generate
training data

19. Experiment
• For the case shown as below, we expect that
red will be the dominant feature if people
selected red scheme as liked group and
scheme mix with blue and green as disliked
group.

20. Experiment
• Result: Red is more dominant than blue and
green; red(0.75)> blue(-0.09)> green(-0.6).

21. Experiment
• For the case shown as below, we expect that
red will be the dominant feature if people
selected blue scheme as liked group and
scheme mix with red and green as disliked
group.

22. Experiment
• Result: Blue is more dominant than blue and
green; blue(0.25)> green(0.046)> red(-0.043).

23. Personal Saliency Map
Personal Saliency Map

24. Personal Saliency Map
1
2
3
…
N
Ranked Grids
Y=0.75R+ -0.09G+ -0.6B
60
50
23
…
33
Original Grids Personal Saliency Map

25. Experiment Result
• Green Dominant

26. Experiment Result
• Red Dominant

27. Experiment Result
• Green Dominant

28. Experiment Result
• Red Dominant

29. Experiment Result
• Green Dominant

30. Experiment Result
• Blue Dominant

31. Experiment Result
• Blue Dominant

32. Experiment Result
• Red Dominant

33. Green dominant
Red dominant

34. Green dominant
Red dominant

35. Thank you for your attention!