This document discusses machine learning best practices and vulnerabilities. It begins by introducing machine learning concepts like the perceptron and different modeling approaches from easy to hard. It then discusses advanced neural network modeling techniques. The document warns that while convolutional neural networks can achieve high accuracy in image classification, they are vulnerable to adversarial examples crafted by intentionally tweaking pixels in ways imperceptible to humans but that cause the model to misclassify. It describes how an attacker can generate adversarial examples step-by-step and provides a defense of retraining models with adversarial examples to increase robustness.

1. Machine Learning
Best practices and Vulnerabilities
MACHINE LEARNING
BEST PRACTICES AND VULNERABILITIES

2. Sebastiano Galazzo
Microsoft MVP
@galazzoseba
sebastiano.galazzo@gmail.com
https://it.linkedin.com/in/sebastianogalazzo

3. Best practices

4. THE PERCEPTRON
WPC2017 4
In machine learning, the perceptron is a binary classifier or a function which can decide whether or
not an input, represented by a vector of numbers, belongs to some specific class.
𝑓 𝑥 = χ ( ⟨ w , x ⟩ + b )
w is a vector having weights of real values, while operator ⟨ ⋅ , ⋅ ⟩ is the scalar product, b is the 'bias’,
a constant not related to any input value and χ ( y ) is the output function

5. MAIN EVOLUTIONS
1. Easy way, Logistic Regression, Support Vector Machine
• Pro: Easy and fast use use
• Cons: Low accuracy (compared to neural networks)
2. Hard way, Neural Networks
• Pro: If get convergence you gain a very high accuracy (State of the art)
• Cons: Very difficult to model, a lot of experience is required
WPC2017 5

6. EASY WAY
• Pseudo equation
• 𝑥 ∗∝ +𝑦 ∗ 𝛽 + 𝑐 ∗ 𝛿+. . +𝑧 ∗ 𝜔 = (0,1)
• #logisticregression #svm
WPC2017 6

7. HARD WAY
• #neuralnetwork
WPC2017 7

8. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 8
Use case, provide a customer's willingness to vote a political party
Age Gender Income City Political party
30 Male 38,000 New York Democrat
39 Female 42,000 Page Republican
24 Other 39,000 San Francisco Republican
51 Prefer not to say 71,000 Seattle Democrat

9. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 9
> 20 parameters
0,17 18,24 25,35 36,45 46,60 > 60 𝑚𝑎𝑙𝑒 𝑓𝑒𝑚𝑎𝑙𝑒 𝑢𝑟𝑏𝑎𝑛 𝑟𝑢𝑟𝑎𝑙 [𝑠𝑢𝑏𝑢𝑟𝑏𝑎𝑛]..[democrat][Republican]
Age Gender Income City Political party
30 Male 38,000 New York Democrat
39 Female 42,000 Page Republican
24 Other 39,000 San Francisco Republican
51 Prefer not to say 71,000 Seattle Democrat

10. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 10
0.42 1 0.5 [0.25][<20.000][21.000-30.000][31.000-40][41.000-60.000]…
Age Gender
/= 100 /4
[0,100] 0 = Male, 0.25 = Female, 0.5=Other, 0.75 = Prefer not to Say, 1 =
Unk
Age Gender Income City Political party
30 Male 38,000 New York Democrat
39 Female 42,000 Page Republican
24 Other 39,000 San Francisco Republican
51 Prefer not to say 71,000 Seattle Democrat

11. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 11
Method 1-of-(C-1) effects-coding: Standard deviation
𝜎 =
1
𝑁
𝑖=1
𝑁
𝑥𝑖 − 𝜇 2
𝜇 = 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑜𝑓 𝑎𝑙𝑙 𝑣𝑎𝑙𝑢𝑒𝑠
Age Gender Income City Political party
30 Male 38,000 New York Democrat
39 Female 42,000 Page Republican

12. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 12
𝜎 =
1
𝑁
𝑖=1
𝑁
𝑥𝑖 − 𝜇 2
age = (30 + 36 + 52 + 42) / 4 =
40.0
𝜎 =
30 − 40 2 + 36 − 40 2 52 − 40 2 42 − 40 2
4
= 8,12
Age Gender Income City Political party
30 Male 38,000 New York Democrat
39 Female 42,000 Page Republican

13. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 13
𝑉′
=
(𝑉 − 𝑚𝑒𝑎𝑛)
𝑠𝑡𝑑 𝑑𝑒𝑣
𝑉′
input will be used in place of the original input
Having the age average is 40.0, standard deviation is 8.12, and our current value is 30.0:
30.0 =
(30 − 40)
8.12
= −1.23
Age Gender Income City Political party
30 Male 38,000 New York Democrat
39 Female 42,000 Page Republican

14. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 14
One of parameters: Italian cities (About 8000)
𝑀𝑖𝑙𝑎𝑛𝑜 𝑇𝑜𝑟𝑖𝑛𝑜 𝑅𝑜𝑚𝑎 … 𝐶𝑎𝑡𝑎𝑛𝑖𝑎
213 = 8192Binary compression:
With 13 nodes we can map 8192 values
(Having the same meaning/context)
City Value
Milano 0,0,0,0,0,0,0,0,0,0,0,0,0,0
Torino 0,0,0,0,1,1,0,0,0,0,0,1,0,0
Catania 0,1,0,0,1,0,0,0,0,1,0,1,1,0

15. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 15
−1,23 1 3,4 [0.25][0,3]
The model has a mapping ratio of 1:1 between concepts and the number of neurons.
Only 5 parameters!
Can be managed without neural networks by an IF,THEN sequence in the code
Age Gender Income City Political party
30 Male 38,000 New York Democrat
39 Female 42,000 Page Republican

16. ADVANCED MODELLING OF NEURAL NETWORKS
WPC2017 16
Data must be manipulated and made understandable by the
machine, not for the humans!

17. Vulnerabilities

18. VULNERABILITIES
WPC2017 18
• Let’s imagine that we run an auction website like Ebay. On our website, we want to
prevent people from selling prohibited items .
• Enforcing these kinds of rules are hard if you have millions of users. We could hire
hundreds of people to review every auction listing by hand, but that would be
expensive.

19. VULNERABILITIES
WPC2017 19
Instead, we can use deep learning to automatically check auction photos for prohibited
items and flag the ones that violate the rules.
This is a typical image classification problem.

20. VULNERABILITIES – IMAGE CLASSIFICATION
WPC2017 20
We repeat this thousands of times with thousands of photos until the
model reliably produces the correct results with an acceptable accuracy.

21. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 21
• Convolutional neural networks are powerful models that consider the entire image
when classifying it.
• They can recognize complex shapes and patterns no matter where they appear in the
image.
• In many image recognition tasks, they can equal or even beat human performance.

22. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 22
With a fancy model like that, changing a few pixels
in the image to be darker or lighter shouldn’t have
a big effect on the final prediction, right?
Sure, it might change the final likelihood slightly,
but it shouldn’t flip an image from “prohibited” to
“allowed”.
“expectations”

23. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 23
It was discovered that this isn’t always true

24. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 24
If you know exactly which pixels to change and exactly how much to change them, you can
intentionally force the neural network to predict the wrong output for a given picture without
changing the appearance of the picture very much.
That means we can intentionally craft a picture that is clearly a prohibited item but which
completely fools our neural network

25. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 25
Why is this?

26. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 26
A machine learning classifier works by finding a dividing
line between the things it’s trying to tell apart.
Right now, the classifier works with 100% accuracy. It’s found
a line that perfectly separates all the green points from the
red points.

27. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 27
But what if we want to trick it into mis-classifying one of
the red points as a green point?
What’s the minimum amount we could move a red point
to push it into green territory?
If we add a small amount to the Y value of a red point
right beside the boundary, we can just barely push it
over into green territory.

28. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 28
In image classification with deep neural networks, each
“point” we are classifying is an entire image made up of
thousands of pixels.
That gives us thousands of possible values that we can
tweak to push the point over the decision line.
If we make sure that we tweak the pixels in the image in a
way that isn’t too obvious to a human, we can fool the
classifier without making the image look manipulated.

29. VULNERABILITIES - CONVOLUTIONAL NEURAL NETWORKS
WPC2017 29
+ =
SquirelPeople

30. VULNERABILITIES – THE STEPS
WPC2017 30
1. Feed in the photo that we want to hack.
2. Check the neural network’s prediction and see
how far off the image is from the answer we
want to get for this photo.
3. Tweak our photo using back-propagation to
make the final prediction slightly closer to the
answer we want to get.
4. Repeat steps 1–3 a few thousand times with the
same photo until the network gives us the
answer we want.

31. VULNERABILITIES
Snippet of a Python script using Keras

32. HOW CAN WE PROTECT OURSELVES AGAINST
THESE ATTACKS?
WPC2017 32
Simply create lots of hacked images and include them in your training data set going
forward, that seems to make your neural network more resistant to these attacks.
This is called Adversarial Training and is probably the most reasonable defense to
consider adopting right now.

33. Thanks!