Paper: http://ceur-ws.org/Vol-2670/MediaEval_19_paper_24.pdf Youtube: https://www.youtube.com/watch?v=keLM9fmKJSI Zhuoran Liu and Zhengyu Zhao, Adversarial Photo Frame: Concealing Sensitive Scene Information of Social Images in a User-Acceptable Manner. Proc. of MediaEval 2018, 27-29 October 2019, Sophia Antipolis, France. Abstract: Personal privacy protection has become more and more crucial in the era of big multimedia data and artificial intelligence. This paper presents our submission to pixel privacy task, where we propose to fool the deep visual classification model that is for recognition of sensitive scenes by adding adversarial frame to the image. Experimental results indicate that our method can achieve strong adversarial effects while maintaining the visual appeal and social function of the transformed images. Presented by Zhengyu Zhao

1. Zhuoran Liu, Zhengyu Zhao
Adversarial Photo Frame:
Concealing Sensitive Scene Information of Social Images in a
User-Acceptable Manner
Radboud University (Netherlands)
RU-iCIS @ Pixel Privacy Task 2019:
MediaEval 2019
10th Anniversary Workshop
27-29 October 2019
EURECOM, Sophia Antipolis, France

2. Related work on fooling CNN classifiers with image transformation
Szegedy, Christian, et al. "Intriguing properties of neural networks.", ICLR (2014).
Kurakin, Alexey, et al. "Adversarial examples in the physical world.", ICLR (2017)
Pixel Perturbations
classifier
perturbations

3. Geometric Transformation
golfcart trailer truck
hotel_outdoor fire_escape
Image Enhancement
bedroom beauty_salon
Style Transfer
[1] Engstrom, Logan, et al. "A rotation and a translation suffice: Fooling cnns with simple transformations." NIPS 2017 Workshop on Machine Learning and Computer Security.
[2] Liu, Zhuoran, and Zhengyu Zhao. "First Steps in Pixel Privacy: Exploring Deep Learning-based Image Enhancement against Large-Scale Image Inference." MediaEval. 2018.
[3] Brugman, Simon, et al. “Exploring Three Views on Image Enhancement for Pixel Privacy.” MediaEval 2018.

4. How to achieve strong fooling
effects and also maintain image
quality?

5. Mimicking Domain-Specific Semantics
“Aadversarial graffiti” fools
the stop sign recognizer.
Owen Wilson=100% Owen Wilson<1%
Motivation
[1] Robust Physical-World Attacks on Deep Learning Visual Classification, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li,
Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, Dawn Song. 2018. The IEEE Conference on Computer Vision and
Pattern Recognition
[2] A General Framework for Adversarial Examples with Objectives. ACM Trans. Priv. Sec. 1, 1, Article 1
Adversarial Photo Frame

6. Pipeline
classifier
perturbations

7. Image examples with full-flexibility (Ff) frame
Image examples with weight-constrained (Wc) frame
width=5 (7.66%) width=10 (15.01%)
width=15
(22.06%)
width=15
(22.06%) width=20 (28.81%)

8. Evaluation and Conclusion
Table 1. Evaluation on the Top-1
classification accuracy and aesthetics
(NIMA) score for our five runs.
• The classification acc. drops a lot.
• NIMA scores of the modified images
are close to the original scores.
• Applying adversarial photo frame
yields higher aesthetics scores.
• Surprisingly, wider frame leads to
higher aesthetics score.
• Our approach has strong fooling effects.
• Maintaining high proportion of the
image content helps.
• Adding domain-specific photo frame
improves image appeal.
• NIMA evaluation model may not well
correspond with human judgement.

9. Future Work
- The perturbations in the frame assemble certain patterns of concept, such as flower and star.
- Other image retouching functions, e.g., mosaic stickers as shown in the figures.

10. MediaEval 2019
10th Anniversary Workshop
27-29 October 2019
EURECOM, Sophia Antipolis, France