1) The document discusses a paper on improving visual recognition systems by leveraging human visual biases and generating images from random features. 2) It describes estimating visual biases from human psychophysics experiments, then using those biases to reconstruct images from random features. The reconstructed images can then be used to train machine learning models. 3) The document outlines experiments showing that incorporating estimated human visual biases into machine learning models, such as SVMs, can help improve visual recognition performance compared to models trained without biases.

1. NIPS2015 reading
Akisato Kimura
1

2. Paper to read
NIPS2015
2

3. 1-page summary
• Improving visual recognition systems with the help of
– image reconstruction from (random) features
– huge amount of human annotations for generated images
3
Image reconstruction
(Computer vision)
Visual bias understanding
(Human psychophysics)
Model training
(Machine learning)

4. Visual biases in human visual systems
4
Canonical views
Preferred ways of
viewing objects
Gestalt law
Tendency to organize
visual elements into
unified groups
[Mezuman+ NPS12]
http://graphicdesign.spokanefalls.edu/tutorials/process/gestaltprinciples/gestaltprinc.htm

5. Classification images
• Finding the internal template in human visual system
that discriminates 2 classes
5
[Ahumada Perception96]
𝑓𝑓 𝒙𝒙; 𝒄𝒄 = 𝒄𝒄⊤ 𝒙𝒙
𝒄𝒄 = 𝝁𝝁𝐴𝐴𝐴𝐴 + 𝝁𝝁𝐵𝐵𝐵𝐵 − 𝝁𝝁𝐴𝐴𝐴𝐴 + 𝝁𝝁𝐵𝐵𝐵𝐵
𝝁𝝁𝐴𝐴𝐴𝐴: Average of stimulus where
• The true class = A
• The predicted class = B

6. Estimating biases in feature spaces
• No real images required
– robust to many issues in dataset bias
– It scales 6
Random feature
Feature inverse
Generated image
Is this television or not?
Yes
No
𝒄𝒄 = 𝝁𝝁𝑌𝑌𝑌𝑌𝑌𝑌 − 𝝁𝝁𝑁𝑁𝑁𝑁Approximate internal template of people
150,000 features
from standard Gaussian
• HOGgles
[Vondrick+ ICCV13] Amazon MT

7. Reconstructing images from features
7
[Weinzaepfel+ CVPR11]
[Vondrick+ ICCV13]
[Kato+ CVPR14]
[Mahendran+ CVPR15]

8. HOGgles
• Paired dictionary learning
– Can be applied to other features such as CNN.
8
[Vondrick+ ICCV13]
𝒚𝒚𝑖𝑖
�𝒙𝒙𝑖𝑖
𝑽𝑽
𝑼𝑼

9. Visualizing biases
9

10. Visualizing biases (cont.)
10

11. Leveraging biases for recognition
• Directly utilizing the visual biases c as a classifier
11
𝑓𝑓 𝒙𝒙; 𝒄𝒄 = 𝒄𝒄⊤ 𝒙𝒙

12. Leveraging biases for recognition
• Directly utilizing the visual biases c as a classifier
12
𝑓𝑓 𝒙𝒙; 𝒄𝒄 = 𝒄𝒄⊤ 𝒙𝒙
Shape is an important bias to discriminate objects in CNN features.

13. Learning with human biases
• Incorporating human biases into learning algorithms
for visual recognition
– SVM with orientation (= bias) constraints
13
Hyperplane for classification
Visual bias
It can be solved as a conic program,
by introducing 𝛼𝛼 satisfying
𝑤𝑤⊤ 𝑤𝑤 ≤ 𝛼𝛼 ≤ 𝑤𝑤⊤ 𝑐𝑐/𝜃𝜃

14. Experiments
14
• The performance is improved as the number of positive samples increases.
• The proposed method (SVM + Human) significantly improves the performance

15. Experiments (cont.)
15
• The proposed method (SVM + bias) MAY help alleviate some dataset bias issues.
SVM only bias only

16. 16