Rock Paper Scissors with MLNET.pptx
•Download as PPTX, PDF•
0 likes•200 views
Slides for my session at Virtual ML.NET Conference about developing an image recognition machine learning model for a rock-paper-scissors mobile game with ML.NET and Xamarin
Recommended
Recommended
More Related Content
Similar to Rock Paper Scissors with MLNET.pptx
Similar to Rock Paper Scissors with MLNET.pptx (20)
More from Luis Beltran
More from Luis Beltran (20)
Recently uploaded
Recently uploaded (20)
Rock Paper Scissors with MLNET.pptx
- 2. Let’s play Rock, Paper, Scissors with ML.NET Luis Beltrán
- 3. Luis Beltrán • Researcher - Tomas Bata University in Zlín, Czech Republic. • Lecturer - Tecnológico Nacional de México en Celaya, Mexico. • Xamarin, Azure and Artificial Intelligence @darkicebeam luis@luisbeltran.mx
- 5. Deep Learning • Deep learning is a subfield of Machine Learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. • It is exceptionally effective in discovering patterns. • Algorithms learn through a multi-layered hierarchy. • If you supply the system with tons of information, it will begin to understand and respond in helpful ways.
- 6. Deep learning has an inbuilt automatic multi stage feature learning process that learns rich hierarchical representations (i.e. features). Low-level features Mid-level features Output (e.g. exterior, interior) High-level features Trainable Classifier
- 7. • Image Pixel Edge Texture Motif Part Object • Text Character Word Word-group Clause Sentence Story • Each module in Deep Learning transforms its input representation into a higher-level one, in a way similar to human cortex. Low Level Features Mid Level Features Output High Level Features Trainable Classifier Input
- 8. Convolutional Layers Filter 1 1 1 1 1 1 0.015686 0.015686 0.011765 0.015686 0.015686 0.015686 0.015686 0.964706 0.988235 0.964706 0.866667 0.031373 0.023529 0.007843 0.007843 0.741176 1 1 0.984314 0.023529 0.019608 0.015686 0.015686 0.015686 0.011765 0.101961 0.972549 1 1 0.996078 0.996078 0.996078 0.058824 0.015686 0.019608 0.513726 1 1 1 0.019608 0.015686 0.015686 0.015686 0.007843 0.011765 1 1 1 0.996078 0.031373 0.015686 0.019608 1 0.011765 0.015686 0.733333 1 1 0.996078 0.019608 0.019608 0.015686 0.015686 0.011765 0.984314 1 1 0.988235 0.027451 0.015686 0.007843 0.007843 1 0.352941 0.015686 0.823529 1 1 0.988235 0.019608 0.019608 0.015686 0.015686 0.019608 1 1 0.980392 0.015686 0.015686 0.015686 0.015686 0.996078 1 0.996078 0.015686 0.913726 1 1 0.996078 0.019608 0.019608 0.019608 0.019608 1 1 0.984314 0.015686 0.015686 0.015686 0.015686 0.952941 1 1 0.992157 0.019608 0.913726 1 1 0.988235 0.019608 0.019608 0.019608 0.039216 0.996078 1 0.015686 0.015686 0.015686 0.015686 0.996078 1 1 1 0.007843 0.019608 0.898039 1 1 0.988235 0.019608 0.015686 0.019608 0.968628 0.996078 0.980392 0.027451 0.015686 0.019608 0.980392 0.972549 1 1 1 0.019608 0.043137 0.905882 1 1 1 0.015686 0.035294 0.968628 1 1 0.023529 1 0.792157 0.996078 1 1 0.980392 0.992157 0.039216 0.023529 1 1 1 1 1 0.992157 0.992157 1 1 0.984314 0.015686 0.015686 0.858824 0.996078 1 0.992157 0.501961 0.019608 0.019608 0.023529 0.996078 0.992157 1 1 1 0.933333 0.003922 0.996078 1 0.988235 1 0.992157 1 1 1 0.988235 1 1 1 1 0.015686 0.74902 1 1 0.984314 0.019608 0.019608 0.031373 0.984314 0.023529 0.015686 0.015686 1 1 1 0 0.003922 0.027451 0.980392 1 0.019608 0.023529 1 1 1 0.019608 0.019608 0.564706 0.894118 0.019608 0.015686 0.015686 1 1 1 0.015686 0.015686 0.015686 0.05098 1 0.015686 0.015686 1 1 1 0.047059 0.019608 0.992157 0.007843 0.011765 0.011765 0.015686 1 1 1 0.015686 0.019608 0.996078 0.023529 0.996078 0.019608 0.015686 0.243137 1 1 0.976471 0.035294 1 0.003922 0.011765 0.011765 0.015686 1 1 1 0.988235 0.988235 1 0.003922 0.015686 0.019608 0.019608 0.027451 1 1 0.992157 0.223529 0.662745 0.011765 0.011765 0.011765 0.015686 1 1 1 0.015686 0.023529 0.996078 0.011765 0.011765 0.015686 0.015686 0.011765 1 1 1 1 0.035294 0.011765 0.011765 0.011765 0.015686 1 1 1 0.015686 0.015686 0.964706 0.003922 0.996078 0.007843 0.019608 0.011765 0.054902 1 1 0.988235 0.007843 0.011765 0.011765 0.015686 0.011765 1 1 1 0.015686 0.015686 0.015686 0.023529 1 0.007843 0.007843 0.015686 0.015686 0.960784 1 0.490196 0.015686 0.015686 0.015686 0.007843 0.027451 1 1 1 0.011765 0.011765 0.043137 1 1 0.023529 0.003922 0.007843 0.023529 0.980392 0.976471 0.039216 0.019608 0.007843 0.019608 0.015686 1 1 1 1 1 1 1 1 1 0 1 0 1 -4 1 0 1 0 Input Image Convoluted Image
- 9. Convolution Input Image Convolved Image (Feature Map) a b c d e f g h i j k l m n o p w1 w2 w3 w4 Filter h1 h2 ℎ1 = 𝑓 𝑎 ∗ 𝑤1 + 𝑏 ∗ 𝑤2 + 𝑒 ∗ 𝑤3 + 𝑓 ∗ 𝑤4 ℎ2 = 𝑓 𝑏 ∗ 𝑤1 + 𝑐 ∗ 𝑤2 + 𝑓 ∗ 𝑤3 + 𝑔 ∗ 𝑤4
- 10. Lower Level to More Complex Features Input Image Layer 1 Feature Map Layer 2 Feature Map w1 w2 w3 w4 w5 w6 w7 w8 Filter 1 Filter 2
- 11. Pooling • Max pooling: reports the maximum output within a rectangular neighborhood. • Average pooling: reports the average output of a rectangular neighborhood. 1 3 5 3 4 2 3 1 3 1 1 3 0 1 0 4 MaxPool with 2X2 filter with stride of 2 Input Matrix Output Matrix 4 5 3 4
- 12. Convolutional Neural Network Feature Extraction Architecture 64 64 128 128 256 256 256 512 512 512 512 512 512 Filter Max Pool Fully Connected Layers Living Room Bed Room Kitchen Bathroom Outdoor Maxpool Output Vector
- 13. Training Deep Learning models using ML.NET
- 14. Input Model Solution structure Nuget Packages Output Model (Prediction)
- 15. Main libraries Paths PrepareSet: Loading input images for training, validation, and testing
- 16. Display information to the Console Input data (images)
- 18. Main program Loading data for supervised learning (images include tags) Training and Validation sets Load pipeline: Images loaded in memory Training options: ImageClassificationTrainer chosen, based on the InceptionV3 architecture Training pipeline: Trying to predict a category Both pipelines are combined
- 19. Perform training Model precision is validated using validation dataset Model Metrics calculated Test the classification model using the new images Prepare new images for validation Export the model Consume the model
- 20. ConsumingModel Load a previously trained classification model and prepare test images that were not used before in the training and validation stages ClassifyImages: Test the model with new images
- 22. Validation Precision metrics Model validation results: Image Actual damage class Prediction
- 23. Image classification results ML Model exported as zip file
- 24. Your platform for building anything .NET Desktop Web Cloud Mobile Gaming IoT AI
- 25. COMPILERS LANGUAGES RUNTIME COMPONENTS LIBRARIES INFRASTRUCTURE .NET STANDARD TOOLS VISUAL STUDIO CODE CLI VISUAL STUDIO VISUAL STUDIO FOR MAC WEB CLOUD MOBILE GAMING IoT AI DESKTOP
- 26. Demo
- 28. Unable to find an entry point named 'TF_StringEncodedSize' in DLL 'tensorflow' “I think ml.net support tensorflow 2.3.1 not yet support 2.4, so you must download SciSharp.TensorFlow.Redist 2.3.1” https://github.com/dotnet/machinelearning-samples/issues/880
- 29. Thank you for your attention luis@luisbeltran.mx
Editor's Notes
- Deep learning can be considered as a subset of machine learning. It is a field that is based on learning and improving on its own by examining computer algorithms. While machine learning uses simpler concepts, deep learning works with artificial neural networks, which are designed to imitate how humans think and learn. Until recently, neural networks were limited by computing power and thus were limited in complexity. However, advancements in Big Data analytics have permitted larger, sophisticated neural networks, allowing computers to observe, learn, and react to complex situations faster than humans. Deep learning has aided image classification, language translation, speech recognition. It can be used to solve any pattern recognition problem and without human intervention. Artificial neural networks, comprising many layers, drive deep learning. Deep Neural Networks (DNNs) are such types of networks where each layer can perform complex operations such as representation and abstraction that make sense of images, sound, and text.
- Low-level features are minor details of the image, like lines or dots, that can be picked up by, say, a convolutional filter (for really low-level things) or (for more abstract things like edges). High-level features are built on top of low-level features to detect objects and larger shapes in the image. Convolutional neural networks use both types of features: the first couple convolutional layers will learn filters for finding lines, dots, curves etc. while the later layers will learn to recognize common objects and shapes.
- Convolution is a general purpose filter effect for images In Convolutional Neural Networks, Filters detect spatial patterns such as edges in an image by detecting the changes in intensity values of the image. In terms of an image, a high-frequency image is the one where the intensity of the pixels changes by a large amount, whereas a low-frequency image is the one where the intensity is almost uniform. Usually, an image has both high and low frequency components. The high-frequency components correspond to the edges of an object because at the edges the rate of change of intensity of pixel values is high.
- Convolution is a simple mathematical operation which is fundamental to many common image processing operators. Convolution provides a way of `multiplying together' two arrays of numbers, generally of different sizes, but of the same dimensionality, to produce a third array of numbers of the same dimensionality.
- 3. Now when you apply a set of filters on top of that (pass it through the 2nd conv. layer), the output will be activations that represent higher-level features. Types of these features could be semicircles (a combination of a curve and straight edge) or squares (a combination of several straight edges). As you go through the network and go through more conv. layers, you get activation maps that represent more and more complex features.
- Pooling layers provide an approach to down sampling feature maps by summarizing the presence of features in patches of the feature map. Two common pooling methods are average pooling and max pooling that summarize the average presence of a feature and the most activated presence of a feature respectively. A pooling layer is a new layer added after the convolutional layer. Specifically, after a nonlinearity (e.g. ReLU) h
- http://www.laurencemoroney.com/rock-paper-scissors-dataset/