This paper proposes an image generation attack that targets image scaling algorithms. The attack aims to 1) modify an image A to appear as image B when A is resized, and 2) introduce minimal distortions such that the attacked image still resembles A. The attack is model-agnostic, as it can target any model using a particular scaling framework and function. The paper develops an optimization approach to craft perturbed images that appear as a target when resized, and demonstrates successful attacks against commercial cloud vision APIs. Potential applications include data poisoning, evasion attacks, and fraud. Detection methods like color histograms may help identify such attacked images.

1. Seeing is Not
Believing: Camouflage
Attacks on Image
Scaling Algorithms
Junyaup Kim
20-03-11

2. TL DR
• This paper suggests image generation algorithm to form as convex optimization
to attack image scaling function. The objective of attack is
1. Make image 𝐴 to 𝐵 when the 𝐴 is resized.
2. The distortion should be small enough that attack image should be almost looks
like 𝐴
• By this, we can assure that this attack is model-free. You can attack any model
that is using certain frame work and certain scaling function. And scaling
function is used in any code line
• They suggests effective querying process to reveal Cloud Vision API provider’s
scaling size

3. Prerequisite
What is scaling?
• Scaling function is resizing function that make input
image to match specific shape.
• Deep learning model is basically matrix calculation.
So we should make our input static (solid shape)

4. Prerequisite
Inconsistency in DL model input shape and camera size
Basic Camera resolution chart Deep learning model input shape
Image scaling function is essential to every deep learning model.

5. Prerequisite
Interpolation and sampling
Bilinear interpolation
Interpolation: A type of estimation, a method of
constructing new data points within the range of a
discrete set of known data points. [Inter + pole]
Linear InterpolationGiven set Spline Interpolation

6. Prerequisite
Interpolation and sampling
Sampling: sampling is the reduction of a
continuous-time signal to a discrete-time signal
Bit depth : Quantization of input signal
Sampling rate : Quantization of time segment

7. Prerequisite
The Nyquist theorem specifies that a sinusoidal function in time or
distance can be regenerated with no loss of information as long as it is
sampled at a frequency greater than or equal to twice per cycle.
Alias and Nyquist Theorem
Nyquist Theorem

8. Prerequisite
Nyquist Theorem
Let’s suppose that pixel values are the discrete signal.
When we scale down the input image, we have not sufficient information of original image
value. To prevent aliasing artifacts, we must use filter to erase the aliasing artifacts.
Without optical low-pass filter With optical low-pass filterWithout optical low-pass filter

9. Prerequisite conclusion
1. Pixels are discrete signal.
2. We need filter with coefficient to scale down the image.
3. Scaling the image can be considered at data under-sampling
4. Due to physical limitations, scaling is used almost every deep
learning model.

10. Main subject
1. Background
• A lot of DL Framework provide
their own image resize
method.
• Order of interpolation is
Horizonal to vertical.(element
wise to channel-wise)

11. Main subject
1. Background
• Even though you are not using
resize function, somewhere in
the framework might inferring
the resize function.

12. Main subject
2. Objective
• The objective of this goal is map
the perturbations on Source
image that after scaling function
𝑆𝑐𝑎𝑙𝑒𝐹𝑢𝑛𝑐(𝑥) , the attack image
turns into target image.
𝑆𝑐𝑎𝑙𝑒𝐹𝑢𝑛𝑐(𝑥)
Source Image
Attack Image Target Image
𝑆𝑐𝑎𝑙𝑒𝐹𝑢𝑛𝑐(𝑥)
Source Image
Attack Image Target image

13. Main subject
3. Taxonomy
• Source image (𝑆 𝑚∗𝑛): the image that an
attacker wants the attack image to look like
• Attack image (𝐴 𝑚∗𝑛): the crafted image
eventually created and fed to the scaling
function
• Output image (𝐷 𝑚′∗𝑛′): the output image of
the scaling function
• Target image (𝑇 𝑚′∗𝑛′): the image that the
attacker wants the outImg to look like
• Scale function (ScaleFunc): The scaling function
of image.
𝑆 𝑚∗𝑛 + ∆1 = 𝐴 𝑚∗𝑛
∆1 = 𝐴 𝑚∗𝑛- 𝑆 𝑚∗𝑛
∆2 = 𝐷 𝑚′∗𝑛′ - 𝑇 𝑚′∗𝑛′

14. Main subject
4. Attack method
• Strong attack form: we KNOW the source image
that wants to make it to attack image
• Weak attack: we DON’T know the source image
that wants to make it to attack image.
Unknown +
Example output image
=

15. Main subject
4.1 Strong attack form
• Strong attack form: we KNOW the source image
that wants to make it to attack image
• Weak attack: we DON’T know the source image
that wants to make it to attack image.
Objective function: min(| 𝐴 𝑚∗𝑛− 𝑆 𝑚∗𝑛 |2
)
Constraints: ||𝑇 𝑚′∗𝑛′ - 𝐷 𝑚′∗𝑛′||∞ ≤ 𝜀 ∗ 𝐼𝑁 𝑚𝑎𝑥

16. Main subject
4.1 Coefficient analysis
• As we said before, we need filter matrix to
resize not to alias the image.
• And because of overlapping in filter
matrix(like CNN) we need to calculate
separately to make perturbation

17. Main subject
4.2 Coefficient analysis
• Horizontal scaling (𝑚 ∗ 𝑛 → 𝑚 ∗ 𝑛′)
• Vertical scaling(𝑚 ∗ 𝑛′ → 𝑚′ ∗ 𝑛′)
Coefficient matrix example
Vertical Horizontal
Craft sequence

18. Main subject
4.3 Strong attack form
• Constraints is a upper boundary of pixel
value(Constant function). So this constraints is
Linear.
• By that, we can calculate this as a convex form
Objective function: min(| 𝐴 𝑚∗𝑛− 𝑆 𝑚∗𝑛 |2
)
Constraints: ||𝑇 𝑚′∗𝑛′ - 𝐷 𝑚′∗𝑛′||∞ ≤ 𝜀 ∗ 𝐼𝑁 𝑚𝑎𝑥
WLOG

19. Main subject
4.4 Strong attack form algorithm analysis
• Decomposition into sub matrix problem.

20. Main subject
4.5 Cloud inference attack(black box)
• We have to know the exact size of
cloud DL model input size.
• inferring model image serach space is
𝑂 𝑁4
= (𝑃𝑎𝑘𝑐𝑎𝑔𝑒 ∗
𝑆𝑐𝑎𝑙𝑖𝑛𝑔 𝑚𝑒𝑡ℎ𝑜𝑑 ∗ ℎ𝑒𝑖𝑔ℎ𝑡 ∗ 𝑤𝑖𝑑𝑡ℎ)
 setting range[201,300] in H, W
 Infer different class by k times at the same
time (k=4)

21. Main subject
5.1 Result
• Attack target: Azure, Baidu, Aliyun, Tencent
• Testing Dataset: 935 (Crafted)
 Class except Sheep or sheep-like animal
 Set as 800*600 image
 𝜀 = 0.01
 Target = Sheep
• Baidu , Aliyun ,Tencent got 100% success ratio
where as Azuzre is more complex
• CDF(cumulative distribution function) shows
that Tag and description is successfully attacked
by this algorithm.

22. Main subject

23. Main subject
5.2 Possible attack scenario
• Data poisoning on database.
• Detection evasion and Cloaking on CNN
based deep learning models.
• Fraud by Leveraging Inconsistencies
between Displays. (ex mobile)

24. Main subject
5.3 Detection of attack
• Color-histogram-based Detection
• Color-scattering-based Detection

25. Conclusion
6. Pros
• This attack is model-free attack. It means
that we can use this attack in any situation
(not only limited in Deep learning)
• This attack is more light-weight than
adversarial attack by deep learning.
• Attack success ratio & confidence is high.
6. Cons
• If the model do not use the resize method(such
as yolo based object detection). It cannot be
successful.
• Only can be applied on smaller attack image
• The perturbations are easily recognizable by
human. The key of this kind of attack is should
be out of human-eye. You can easily recognize
that this image is somewhat wrong.