Machine Learning for iOS developers
•Download as PPTX, PDF•
0 likes•443 views
This document summarizes a talk by Keith Moon on machine learning for iOS developers. It discusses three options for adding machine learning features to iOS apps: 1) using a managed ML API, 2) training and using a custom model on a server, or 3) training a model on a server and using it on a phone. The talk provides an overview of machine learning concepts and tools for iOS developers and explores the current and future state of machine learning on iOS.
Recommended
More Related Content
Similar to Machine Learning for iOS developers
Similar to Machine Learning for iOS developers (20)
Recently uploaded
Recently uploaded (20)
Machine Learning for iOS developers
- 1. Keith Moon - Senior iOS Developer Machine Learning for iOS Developers UnitedDefConf - Minsk April 2017
- 2. 2 • iOS Developer since 2010 • Worked with BBC News, Hotels.com and Travelex • “Swift 3 Cookbook” to be published by Pakt Who am I? @keefmoon
- 3. 3 • Help users discover great local food • Make it quick and easy to order from a wide variety of takeaways • Available on: –Web –iOS –tvOS –Android –Amazon Echo What is Just Eat? 3
- 4. 4 • Australia • Brazil • Canada • Denmark • France • Ireland • Italy • Mexico What is Just Eat? Global Business • New Zealand • Norway • Spain • Switzerland • UK
- 5. 7 What this talk isn’t? • Not a ML expert • Not a Python expert • Not great at Maths • Not a deep dive into ML So what is this talk?
- 6. So what is this talk? 8 • ML from an iOS developer’s point of view • High level overview of ML • What features are appropriate uses of ML? • How can you use ML in your apps? • Current state of ML tools • What does the future hold?
- 7. Glossary 9 Machine Learning Artificial Intelligence Neural Network Deep Learning
- 8. Glossary 10 Machine Learning Artificial Intelligence Neural Network Deep Learning
- 9. Artificial Intelligence Glossary 11 Machine Learning Neural Network Deep Learning
- 10. Artificial Intelligence Glossary 12 Machine Learning Neural Network Deep Learning
- 11. Artificial Intelligence Glossary 13 Machine Learning Deep Learning Neural Network
- 13. ML Use Cases 15 • Spam • Recommendations • Handwriting recognition • Speech recognition • Face Detection • Entity extraction • Facial Recognition • Object Recognition • Text Prediction • Sentiment Analysis • Image Style transfer
- 14. Machine Learning Goal 16 Neural Network Output Answer Training Input Input Question
- 15. Classifier 17 Untrained Neural Network Training Input Data Data Data Input to categorise Category 1 Category 2 Category 3 1 1 2 2 3 3
- 16. Classifier 18 Trained Neural Network Training Input Input to categorise Category 1 Category 2 Category 3
- 17. Classifier 19 Neural Network Input Input to categorise Hidden Training Input Data Data Data 1 2 3 Category 1 Category 2 Category 3 1 2 3 Output
- 18. ⨍( ) Training the model 20 w1 w2 w3 ∑i xi * wi + bx1 x2 x3 b Forward Propagation eg. Softmax
- 19. Training the model 21 w1 w2 w3 x1 x2 x3 b Back Propagation g = gradient The extent to which changing the value reduces the error g Difference between expected and actual = error g g g
- 20. Classifier 22 Neural Network Input Input to categorise Hidden Training Input Data Data Data 1 2 3 Category 1 Category 2 Category 3 1 2 3 Output
- 21. Handwriting Image Input 25 Input ? ? ? ? from the MNIST dataset
- 22. Handwriting Image Input 26 Input ? ? ? ? 14 pixels x 14 pixels 14 x 14 = 196 values between 0 and 1
- 23. Handwriting Image Input 27 Input 14 pixels x 14 pixels 14 x 14 = 196 values between 0 and 1 … px 1 px 2 px 3 px 4 px 5 px 6 px 196
- 24. Your App API Adding a ML Feature 28 What are my options? Train Use 1) Managed Machine Learning driven API
- 25. 29 • No ML knowledge required • Simple to implement • Light on resources • Only solves common ML problems • Third-Party dependancy • Needs connectivity • Data ownership issues • No control over model improvement Adding a ML Feature What are my options? 1) Managed Machine Learning driven API
- 26. 30 Adding a ML Feature What are my options? 2) Custom Model Trained and Used on Server Your App API Train Use
- 27. 31 • Can customise the model to your needs • Make use of open source models • Light on mobile resources • You control the data • You control model improvement • Knowledge of ML tools / Python needed • Server management overhead • App friendly API needed • Needs connectivity Adding a ML Feature What are my options? 2) Custom Model Trained and Used on Server
- 28. 32 Adding a ML Feature What are my options? 3) Custom Model Trained on Server. Used on Phone. API Train Trained ModelAccelerate Framework Metal Framework GPU CPU
- 29. 33 • Can customise the model to your needs • Make use of open source models • User can control the data • Can work offline • Knowledge of ML tools / Python needed • Server management overhead • Need to transfer complex model to phone • May limit the scope of model improvement Adding a ML Feature What are my options? 3) Custom Model Trained on Server. Used on Phone.
- 30. 35 Adding a ML Feature What are my options? 3) Custom Model Trained on Server. Used on Phone. 2) Custom Model Trained and Used on Server 1) Managed Machine Learning driven API
- 31. Future of Machine Learning on iOS ● Easier model transfer from server to phone ● Trainable network APIs from Apple ● Ability to plug and play ML models together ● Further development of Swift ML tools 36
- 32. References Machine Learning APIs: Google Prediction: https://cloud.google.com/prediction Google Natural Language: https://cloud.google.com/natural-language Microsoft Cognitive Services: https://www.microsoft.com/cognitive-services Amazon ML: https://aws.amazon.com/documentation/machine-learning IBM Watson: https://www.ibm.com/watson/developercloud Open Source Model: Tensor Flow Models: https://github.com/tensorflow/models FaceNet for TensorFlow: https://github.com/davidsandberg/facenet 37
- 33. References Machine Learning Frameworks: Torch: http://torch.ch TensorFlow: https://www.tensorflow.org Caffe: https://github.com/BVLC/caffe Awesome Machine Learning resources: https://github.com/josephmisiti/awesome-machine-learning Hosting: Amazon Web Services: https://aws.amazon.com Amazon Deep Learning AMI - Ubuntu Edition https://aws.amazon.com/marketplace/pp/B06VSPXKDX Digital Ocean https://www.digitalocean.com 38
- 34. References Machine Learning Frameworks on iOS: Torch: https://github.com/clementfarabet/torch-ios TensorFlow: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/ios_examples http://www.mattrajca.com/2016/11/25/getting-started-with-deep-mnist-and-tensorflow-on-ios.html Caffe: https://github.com/noradaiko/caffe-ios-sample Using Metal Performance Shaders with a TensorFlow trained model: https://developer.apple.com/library/content/samplecode/MPSCNNHelloWorld Neural Networks and Accelerate: https://developer.apple.com/videos/play/wwdc2016/715 BNNS in Accelerate: https://developer.apple.com/reference/accelerate/bnns List of ML resources for iOS: https://github.com/alexsosn/iOS_ML 39
- 35. References Open Source: Face Entry Example: https://github.com/keefmoon/faceentry Just Eat Open Source: https://github.com/justeat 40
Editor's Notes
- Quickly demo MSQRD Talk about face detection, position, rotation in 3D space
- Select use cases are classification problems. They take a given input and the problem involves deciding which category to put the answer in, and how well do they fit into that category. There is finite number of the categories. For spam is either spam, or not spam For handwriting recognition the category is a letter or word, for facial recognition it is a specific person.
- Lets look at how would use a neural network to do machine learning. We want to train a neural network to get good at turning inputs into outputs with known data, so that it can use do the same with inputs where we don’t know the expected output. What does that mean, to train the network?
- Let’s talk specifically about using a neural network to work on a classification problem. We feed data into one end of the neural network, and we expect those inputs to produce an expected output category, if it doesn’t, we need to neural network until it does. Do that with enough data, and the network becomes good at putting the right type of data in the right categories.
- Now that we have a trained network, we put in data where we don’t know the category it belongs in, and the network uses the accumulated information from it’s training to produce the right category as an output. Ok, but what is a neural network’s structure?
- Like any other network, a neural network is made up of nodes and connections. Neural Networks are organised in layers. Often with all the nodes in one layer connected to all the nodes in the next layer. The input layer receives in input data, we will get to how that happens soon. Information then passes to a number of hidden layer. Called hidden because they aren’t related to input of output. Deep learning, a term we discussed early, refers to how deep the network is, or how many hidden layers there are.
- Define: Model - The structure of the neural network Trained Model - The structure + the weights and bias’ determined during training
- Define: Model - The structure of the neural network Trained Model - The structure + the weights and bias’ determined during training
- Like any other network, a neural network is made up of nodes and connections. Neural Networks are organised in layers. Often with all the nodes in one layer connected to all the nodes in the next layer. The input layer receives in input data, we will get to how that happens soon. Information then passes to a number of hidden layer. Called hidden because they aren’t related to input of output. Deep learning, a term we discussed early, refers to how deep the network is, or how many hidden layers there are.
- Define: Model - The structure of the neural network Trained Model - The structure + the weights and bias’ determined during training
- Show FaceEntry to explain input preparation
- Define: Model - The structure of the neural network Trained Model - The structure + the weights and bias’ determined during training
- Define: Model - The structure of the neural network Trained Model - The structure + the weights and bias’ determined during training
- Define: Model - The structure of the neural network Trained Model - The structure + the weights and bias’ determined during training
- Show FaceEntry as an example
- Show MNIST on AWS as example