Paper: http://ceur-ws.org/Vol-2882/paper16.pdf YouTube: https://youtu.be/ix_b9K7j72w Zhengyu Zhao : Fooling Blind Image Quality Assessment by Optimizing a Human-Understandable Color Filter. Proc. of MediaEval 2020, 14-15 December 2020, Online. This paper presents the submission of our RU-DS team to the Pixel Privacy Task 2020. We propose to fool the blind image quality assessment model by transforming images based on optimizing a human-understandable color filter. In contrast to the common work that relies on small, $L_p$-bounded additive pixel perturbations, our approach yields large yet smooth perturbations. Experimental results demonstrate that in the specific context of this task, our approach is able to achieve strong adversarial effects, but has to sacrifice the image appeal. Presented by: Zhengyu Zhao

1. Zhengyu Zhao
Fooling Blind Image Quality Assessment by
Optimizing a Human-Understandable Color Filter
Radboud University (Netherlands)
RU-DS @ Pixel Privacy Task 2020:

2. Fooling ML models with small, imperceptible perturbations
Szegedy et al. "Intriguing properties of neural networks.", ICLR 2014.
Kurakin et al. "Adversarial examples in the physical world.", ICLR 2017.
Original
CNN CNN
Perturbations Adversarial
vulnerable to image processing
(e.g. JPEG compression)

3. [1] semantic manipulation [3] DL-based colorization
golf-cart
[2] spatial transformation
trailer truck
domain-specific
costly and
not human-understandble
limited action space
[1] Qiu et al. "SemanticAdv: Generating Adversarial Examples via Attribute-conditioned Image
Editing." ECCV 2020.
Eykholt et al. "Robust physical-world attacks on deep learning visual classification." CVPR 2018.
[2] Engstrom et al. "Exploring the landscape of spatial robustness." ICML 2019.
[3] Bhattad et al. "Unrestricted adversarial examples via semantic manipulation." ICLR 2020.
Fooling ML models with large yet unnoticeable perturbations

4. [1] Zhao et al. "Adversarial Robustness Against Image Color Transformation within Parametric
Filter Space." Under review. Preliminary work at BMVC 2020.
[2] Hu et al. "Exposure: A white-box photo post-processing framework." ACM TOG 2018.
[3] Kurakin et al. "Adversarial examples in the physical world." ICLR 2017.
[4] Carlini et al. "Towards evaluating the robustness of neural networks." In IEEE S&P 2017.
Simple and human-understandable filter [2]
Generally applicable to all kinds of color images
Relatively large action space (hundreds of parameters)
C&W [4]:
ACE-Ins:
2
)
,
(
min 




 y
x
L
ε
y
x
F
L 




s.t.
),
,
)
(
(
min 2
)
(
)
),
(
(
min ins
x
x
F
y
x
F
L 
 



ε
y
x
L 
 



s.t.
),
,
(
min
PGD [3]:
ACE-PGD:
Our Adversarial Color Enhancement (ACE) [1]

5. Results
Half of the images to be predicted as low-quality
Robust against JPEG compression
Unacceptable image appeal

6. Examples

7. (a) PGD (b) Colorization (c) ACE-PGD
ACE-Ins
Examples of fooling an ImageNet classifier
Guessing:
BIQA model tends to rely on high-frequency features for quality assessment
→ Robust against low-frequency perturbations by ACE
ImageNet classifier learns both low- (e.g. shape) and high-frequency (e.g.
textures) features for object classification
→ Less Robust against low-frequency perturbations by ACE