The document provides an overview of machine learning from a software engineer's perspective. It discusses how machine learning can solve various problems, introduces TensorFlow as an open-source machine learning library, and provides an example of using TensorFlow to build a neural network model for handwritten digit classification that achieves 97.5% accuracy on MNIST data. The document emphasizes that machine learning requires examples to learn from and recommends starting simply before gradually increasing complexity when experimenting with neural networks.

1. Machine learning from a
software engineer’s perspective
Marijn van Zelst
AMSTERDAM 16 - 17 MAY 2017

2. About me
@marijnvanzelst
Marijn van Zelst
• Software engineer at Luminis Amsterdam

3. Goal:
• Practical knowledge about machine learning
• What kind of problems it can solve
• Do your own experiment

4. Why is machine learning
interesting for Software Developers?
• New opportunities
• Builds on existing knowledge
• Open-source communities and libraries

5. Tensorflow
• Developed by Google
• Open-source
• Has api’s for most major programming languages
• High level api

6. Machine learning
Learn from examples

7. Handwritten digit classification
MNIST dataset

8. What is the difference between a
2 and a 3?

9. Neural networks
Image recognition
Speech recognition
Natural language recognition
Churn prediction
Fraud detection
Anomaly detection

10. Neural networks
Pixel 1
Pixel 2
Pixel 3
Pixel 784
Zero
Two
One
Nine

11. Neurons
x3
x1
x2
w1
w2
w3 b
((x1 * w1) + (x2 * w2) + (x3 * w3)) * b

12. Activation function
x3
x1
x2
w1
w2
w3
((x1 * w1) + (x2 * w2) + (x3 * w3)) * b
b
Input
Output (activation)

13. Learning algorithm
• While not done
• pick a training example (input data, class)
• run it through the neural network which produces a prediction
• modify the weights in the neural network so that the prediction is closer to the actual class

14. Agile approach to machine learning
• Theory takes a lot of time to explain and learn
• Using high level api’s you can do experiments without knowing all the mathematical details
• When you have your first results and want to improve
• Learn more
• Ask for expert help

15. Using a high level api
• Which features do I use?
• Do I need more data?
• How many neurons and layers?
• How many training cycles?

16. Distinguish 0s and 1s

17. Just two features
blackwhite

18. Two dimensional feature space
white black
black 0
1
y
x
y
x
y
x

19. white black
black 0
1
y
x

21. white black
black
y
x
x
y

22. w1
w2 b
w1=0.7
w2=0.62
b=0.085

23. w1
w2 b
w1=0.7
w2=0.62
b=2
w1=0.7
w2=0.62
b=-1
w1=0.5
w2=0.2
b=0.085
w1=0.2
w2=0.5
b=0.085

24. Neural nets intuition
• Input:
• Features (coordinates in feature space)
• Output:
• A predicted class at every coordinate
in the feature space
white black
black
y
x

25. More training examples
x
y
x
y
x
y
x
y

26. Two dimensional feature space
white black
black 0
1
y
x
y
x
y
x
0 & 1

27. Neural nets intuition
• The combination of several features make classes distinguishable
• Training on more examples doesn’t always work
• Neural networks can handle thousands of features
white black
black 0
1
y
x
0 & 1

28. white black
black
y
x
x
yx y
x
y
x
y

29. y
x
y
x

30. Neural nets intuition
Adding a feature can make other features more useful

32. 50 steps 600 steps5 steps
train - 72.5% train - 88.8% train - 92.2%

34. 50 steps 600 steps5 steps
train - 72.5% train - 88.8% train - 92.2%
test - 76.5% test - 82.6% test - 80.3%

36. Neural nets intuition
• Keep a test set to verify the accuracy of your training algorithm
• Keep an eye on the test set accuracy during training to decide when to stop

37. 78% 88% 71%

38. 78%88%
71%

39. Neural nets intuition
• How many neurons?
• Depends on the complexity of the data
• Start small, gradually ramp up
• Check for overfitting

40. Insights
• High dimensional
• Can handle and find correlations between lots of features
• They do probabilistic predictions
• They require a training set and a test set for validation

41. from tensorflow.contrib.learn import infer_real_valued_columns_from_input, DNNClassifier
from tensorflow.examples.tutorials import mnist
mnist_data = mnist.input_data.read_data_sets("MNIST_data/")
train_data, test_data = mnist_data.train.images, mnist_data.test.images
train_labels, test_labels = mnist_data.train.labels.astype('int32'),
mnist_data.test.labels.astype('int32')
estimator = DNNClassifier(
feature_columns=infer_real_valued_columns_from_input(mnist_data.train.images),
hidden_units = [2500, 1000, 1500, 2000, 500],
n_classes = 10)
estimator.fit(x = train_data, y = train_labels, steps = 1000)
print(estimator.evaluate(x = test_data, y = test_labels)['accuracy'])
# 97.5% accuracy in ~15min
Handwritten digit classifier in TensorFlow

42. Conclusion
• You can use machine learning using a high level api
• It can make complex decisions on lots of features
• Using high level api’s you can do experiments without knowing all the mathematical details
• When you have your first results and want to improve
• Learn more
• Ask for expert help

43. Resources on machine learning
• Tensorflow high level api quickstart: https://www.tensorflow.org/get_started/tflearn
• Other resources:
• My blog on the text classification with some more in depth code: https://github.com/luminis-ams/
blog-text-classification
• For a theoretical understanding of the backpropagation algorithm: Machine learning course
Coursera - Andrew Ng
• More practical deep learning course: Udacity deep learning course
• Visualization of a neural network (used in the presentation)
• http://playground.tensorflow.org

44. Dijkstra probably hates me - Linus
Torvalds, in kernel/sched.c
?Thank you
Any
Questions?