This paper describes privacy protection method based on a false coloring approach for Drone Protect Task of MediaEval 2015. The aim is to obscure regions of a video that are privacy sensitive without sacrificing intelligibility and pleasantness. False coloring transforms the original colors of pixels using a color palette into a different set of colors in which private information is harder to recognize. The method can be applied globally to an entire frame of the video or to a specific region of interest (ROI). The privacy protected output is expected to remain pleasant, and when needed, a close approximation of the original input can be recovered. Benchmarking evaluations on the mini-drone dataset show promising results, especially, for intelligibility and pleasantness criteria. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

1. MediaEval 2015 Drone Protect Task:
Privacy Protection in Surveillance Systems
Using False Coloring
Serdar Çiftçi1, Pavel Korshunov2, Ahmet O˘guz Akyüz1, Touradj
Ebrahimi2
1. Middle East Technical University, Ankara, Turkey
2. Ecole Polytechnique Fedéralé de Lausanne, Switzerland
MEDIAEVAL 2015 Drone Protect Task
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 1 / 18

2. Outline
1 Introduction
2 Related Work
3 False Color Based Privacy Protection
4 Experiments
5 Conclusion
6 References
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 2 / 18

3. Introduction
Video Surveillance Systems Surround Us
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 3 / 18

4. Related Work
Most Used Methods
Figure taken from [KMDE13].
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 4 / 18

5. Related Work
Warping
Figure taken from [KE13b].
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 5 / 18

6. Related Work
Morphing
Figure taken from [KE13a].
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 6 / 18

7. Related Work
Scrambling
Figure taken from [DE08].
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 7 / 18

8. Related Work
In-Painting
Figure taken from [CVP+
09].
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 8 / 18

9. False Color Based Privacy Protection
Privacy Protection in Surveillance Systems Using False Coloring
Why we need a novel method?
Most methods are computer vision techniques dependent.
There is a trade off between privacy protection and intelligibility.
Reversibility may not be possible.
Pleasantness is an important issue.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 9 / 18

10. False Color Based Privacy Protection
Privacy Protection in Surveillance Systems Using False Coloring
The main steps are as follows:
Convert a color frame into grayscale.
Pixel intensities of the grayscaled frame are then used as keys to
a look-up table that represents a color palette.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 10 / 18

11. False Color Based Privacy Protection
Privacy Protection in Surveillance Systems Using False Coloring
Reverse function steps are as following:
Perform an inverse table look-up.
Reversion is only possible if one knows the properties of the color
map used during protection.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 11 / 18

12. Experiments
Used Color Palettes
LOCS RBS
DEF
LOCS: Linearized optimal color scale (for Low Level Privacy)
RBS: Rainbow color scale (for Medium Level Privacy)
DEF: Radiance default color scale (for High Level Privacy)
Selection of the color palettes are based on our previous work published at SPIE Human Vision
and Electronic Imaging XX 2015 conference.
Using false colors to protect visual privacy of sensitive content,
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 12 / 18

13. Experiments
Example Results
Figure : False color results, a) original frame, c) protected frame, e)
recovered frame.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 13 / 18

14. Experiments
Subjective Evaluation Results
Privacy Intelligibility Pleasantness
FC AVG FC AVG FC AVG
Expert 0.39 0.49 0.76 0.59 0.73 0.60
Naïve 0.34 0.48 0.75 0.58 0.75 0.61
Average 0.365 0.49 0.755 0.59 0.74 0.60
Table : Evaluation results where FC and AVG respectively stand for false
color results and the average of all submissions. Expert and Naïve’s are
participant groups which represent the people conducting research on visual
privacy protection and naïve observers.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 14 / 18

15. Conclusion
Findings and Future Work
We described a simple and effective method for protecting privacy
using false coloring.
Benchmarking evaluations indicated higher than average score for
pleasantness and intelligibility.
However, the privacy protection level was found to be lower than
average.
Future work will investigate designing custom color scales to
improve privacy protection and the quality of reversibility while
enhancing security.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 15 / 18

16. Conclusion
Thank you!
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 16 / 18

17. References
References I
[CVP+09] S-CS Cheung, MV Venkatesh, JK Paruchuri, J Zhao, and
T Nguyen, Protecting and managing privacy information in
video surveillance systems, Protecting Privacy in Video
Surveillance, Springer, 2009, pp. 11–33.
[DE08] Frederic Dufaux and Touradj Ebrahimi, Scrambling for
Privacy Protection in Video Surveillance Systems, IEEE
Trans. on Circuits and Systems for Video Technology vol.
18 (2008), no. no. 8, 1168–1174.
[KE13a] Pavel Korshunov and Touradj Ebrahimi, Using Face
Morphing to Protect Privacy, IEEE International
Conference on Advanced Video and Signal-Based
Surveillance (AVSS), 2013.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 17 / 18

18. References
References II
[KE13b] , Using Warping for Privacy Protection in Video
Surveillance, 18th International Conference on Digital
Signal Processing (DSP), 2013.
[KMDE13] Pavel Korshunov, Andrea Melle, Jean-Luc Dugelay, and
Touradj Ebrahimi, A framework for objective evaluation of
privacy filters in video surveillance, Proceedings of SPIE
Volume 8856 (San Diego, California, USA), Spie-Int Soc
Optical Engineering, 2013.
Serdar Çiftçi, Pavel Korshunov, Ahmet O˘guz Akyüz, Touradj Ebrahimi (METU-EPFL)False Coloring for Visual Privacy September 15th 2015 18 / 18