Downloaded 11 times
Uploaded byMatt Mongeau
Is it food?
The document discusses machine learning and artificial neural networks. It describes how to use transfer learning with Inception V3 to retrain an image classifier on a custom dataset to identify if images contain food. The author details the process of gathering training data, retraining the model with TensorFlow, and serving predictions through a web application built with Flask.
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
Is it food?
- 1. Is it food ajourney of minds, machines, and meals
- 2. @halogenandtoast Who am I? Matthew“Goose” Mongeau
- 3.
- 7.
- 14. Is it food ajourney of minds, machines, and meals
- 15. Why give thistalk?
- 17. Gradient descent isa first-order iterative optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point.
- 18. Gradient descent isa first-order iterative optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point. ಠ_ಠ
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25. Is it aphotograph of food?
- 32.
- 33.
- 34.
- 35.
- 36.
- 37.
- 40.
- 41.
- 42.
- 43.
- 44. A Brief Historyof Artificial Neural Networks
- 45.
- 46.
- 47.
- 48. A Brief Historyof Artificial Neural Networks Finished
- 49.
- 50.
- 52. What is aTensor?
- 58.
- 59. What is aInception?
- 60.
- 62. What is aInception?
- 63.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78. brew install python pipinstall --upgrade virtualenv virtualenv --system-site-packages targetPath cd targetPath source bin/activate pip install --upgrade tensorflow
- 79.
- 80.
- 81.
- 82. Gather up yourtraining data in a folder called data
- 85. curl https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/examples/image _retraining/retrain.py > retrain.py Ichanged output_graph and output_labels from tmp to the current directory python retrain.py --image-dir data
- 86.
- 87.
- 88.
- 89.
- 92.
- 93.
- 94.
- 95. https://codelabs.developers.google.com/cod elabs/tensorflow-for-poets/#4 copy to label_image.py changegraph and labels to local directory
- 98. Making a webserver
- 99.
- 100. Convert our label_image.py intoa small server
- 102.
- 103. Suggested Reading Demystifying DeepNeural Nets - Rosie Campbell Python Machine Learning - Sebastian Raschka
- 104.
- 105. • https://www.flickr.com/photos/su-may/7416737880/in/photolist-cioHM3-5Br3rt-5Br3ok-azvaWq-f8sGux- 6gk5Am-j3FST-Qmwabr-5YFu4j-9U7YHe-8yXv8U-5Br4h8-8KCB4N-fdMBwB-xpVTew-h1UBMH-aFRh8e- dTXMzJ-3JEPYY-6ZujYn-dTSa1i-BGK2B-8QaxYe-9XsJWg-EMsLEu-b1YJDM-buKEvA-jECV3W-autnBw- 8KCzMW-cioHBQ-cwCP6q-oZufBa-6KXrdr-auHvXG-yHNtk-dwG2AF-dTS9HV-e9fPXB-f8sPAt-8HLqym- dTS9jg-py7en9-rykevE-8yvFLb-KQnns-4YWwK7-qLgCEQ-LFUZY-8KCAkN • https://www.chiefoutsiders.com/hubfs/low-hanging-fruit.jpg •http://s1036.photobucket.com/user/abby789_2010/media/DSCN8793.jpg.html • https://www.deltaco.com/files/menu/item/thumb_regulartaco.jpg • https://thenounproject.com/term/test-tube/13589/ • https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html • http://stats.stackexchange.com/questions/203872/what-to-do-when-a-linear-regression-gives-negative- estimates-which-are-not-possi • https://en.wikipedia.org/wiki/Tensor • http://blog.mpacula.com/2011/04/27/k-means-clustering-example-python/ • http://www.bikemaui.com/wp-content/uploads/2016/08/angry-mongoose.jpg • http://www.eltorocyclingclub.nl/wp-content/uploads/2015/05/Gans.jpg
- 106.
- 107.
Editor's Notes
- #2 TEST, if I can see this I’m in luck… this time.
- #16 I wanted to learn more about machine learning. The talks I watched on machine learning were either way too high level or way too low level for me
- #17 I see lots of charts that look like this and talk about things like Linear Regression, Gradient Descent, etc.
- #18 With definitions that look like this.
- #19 And this summarizes how I feel
- #20 I have two main goals with this talk
- #21 I like practicality, so I want to take a problem, solve it, and share the experience so others can do it as well
- #22 And I want to show something in the context of a real world use case. Really by the end of my talk, I hope some audience members go “Oh, that’s neat, looks easy, I’ll try it out myself”
- #23 I’m going to do my best to balance between practical and technical and as a result I might end up oversimplifying some stuff. To be honest this is meant to be a really high level overview. Hopefully, this will coincide with my goals to keep things practical.
- #24 Let’s start with a brief experiment
- #25 I want you all to tell me whether or not something is food.
- #26 Or more specifically, “Is it a photographer food” since some people are quick to point out fallacy of my question. Easy enough right? Let’s hope this will be a fruitful experiment
- #27 Is this food? Well of course it is, It’s a fruit and therefor food. But how do we truly know. For many of us, the answer is simple, we’ve probably eaten an apple before. But that doesn’t really get to the meat of our problem now does it
- #28 Is this food? I speculate that not everyone here has specifically eaten Yakitori, but it’s fairly likely you’ve eaten something similar to it. Therefor we can probably easily identify this as food, and most likely we all instantaneously recognized this as food. Pretty sweet huh?
- #29 Is this food? I think we’d all say yes, but this represents an interesting edge case. No food is actually visible in this picture and yet when I see this I immediately associate it with food. However,
- #30 When I see this, I’m just a little sad. But I also don’t think of this as food, clearly when I see this I think of garbage. And again this is an interesting edge case. Alright just a couple more examples.
- #31 Okay nothing really new here, I’m pretty sure we’ve all a hard shelled taco before, and therefore can clearly identify this as food, but what about
- #32 this.
- #33 I really hope no one said yes
- #34 This is many ways was core to our survival. Being able to see these patterns and take specific actions in response. It’s one of the key factors that has let us develop language, develop tools, let us escape predators, etc. But while we’re exceptional at it, we’re slow, and we hate menial tasks
- #35 So like many problems we want to automate this kind of problem. But automating is difficult because as humans we can recall information in an extraordinary fuzzy way and make similarities between past experiences to extract new information, but programming this kind of process is very difficult
- #36 Machine learning can be used as an approximation of this kind of behavior.
- #38 Clustering, for example if you have an image we can group colors based on their proximity to determine the dominant color in an image.
- #41 Classification, we provide buckets or categories for data to fall as well as examples to train with in order to classify some new data into a category.
- #42 Train as you go, provide some positive effect that you feed back into the system so it can improve over iterations
- #45 Using Machine learning is one means of solving this problem
- #46 The perceptron algorithm was invented in 1957 by Frank Rosenblatt. This machine was designed for image recognition. One of the problems with this style of image recognition was that it couldn’t learn an XOR function and so it could only learn linearly separable patterns.
- #47 This problem actually caused research and development of neural networks to stagnate quite a bit. However, let’s fast forward to 2015.
- #48 In 2015, we get the first public release of TensorFlow.
- #49 And thus ends a brief history of artificial neural networks
- #51 Developed by Google as a system capable or building and training Neural networks which are represented with something called a Data Flow Graph.
- #52 Which might look something like this. Each layer in this graph takes in a tensor and returns a tensor. Performing some operation on the tensor.
- #54 Looking at this image, one would assume you the answer is simply “You will never know”, but the simple answer is that a Tensor is just an n-dimensional array of values.
- #55 Which might look something like this. Each layer in this graph takes in a tensor and returns a tensor.
- #56 Which might look something like this. Each layer in this graph takes in a tensor and returns a tensor.
- #57 Which might look something like this. Each layer in this graph takes in a tensor and returns a tensor.
- #58 Now this particular Data Flow Graph represents a trained Neural Network which has been labeled Inception.
- #59 Specifically Inception V3
- #61 Inception is a pre built data flow graph useful for categorizing images
- #62 Specifically it was designed to categorize image from ImageNet. <<read description>>. You can also download whole sets of data from ImageNet which you could use for training purposes yourself.
- #64 Let’s return to our Problem, is it food. I mentioned at the start of my talk that I wanted to focus on a real world use case.
- #65 At Cookpad where I work, we have this exact problem. We want to be able to tell which photos are actually photos of cooked recipes.
- #66 We want to do this for a number of reasons. First we want to make sure the content we’re showing is what the user expects to see. Users love to break systems, and love to be malicious
- #67 We need to protect ourselves and our users from these kinds of abuses of the system.
- #68 So I want to recreate some of the functionality that system employs. When I started doing this I decided I was going to build a rails app that could do this, but I made a few mistakes along the way.
- #72 I started by trying to user tensorflow.rb, but I wasn’t able to get it to properly build on my machine running OSX. My guess is this is an issue with clang, but I didn’t have any luck trying to compile it with GCC either. One of the suggestions given in the README is to use Docker.
- #74 After setting up the docker image, I tried to compile a program that would let me retrain Inception V3 for my image set, but I couldn’t get that program to compile either. After many attempts I ran into my favorite “Docker” issue.
- #76 So I’d like to talk about “My” road to success with one small note
- #78 So let’s start with installation. Getting everything setup can actually be pretty simple.
- #79 Use python. <<speak about why python/ruby>>.
- #80 The next step was to take Inception and figure out how to retrain it.
- #81 In order to train imagenet, we want to do something called transfer learning. This is where we reuse most of what the neural network has already learned, and we just retrain the last layer.
- #82 In order to do this, we need a bunch of images for the cases in our problem. This was easy because I was able to just reuse the images that my company used, but if you need to collect your own images it could take a while. To be honest this is probably the hardest part.
- #84 Your data set is extremely important. You not only need good examples for things that match your desired categories, but you also need good examples of things that aren’t. If all you’ve ever seen is food, then everything looks like food.
- #85 Each folder has between 1000 to 2000 images (except text which has around 600). The images can be small since inception will end up resizing them to a maximum of 299x299px before running any operations on the image. If you have non-square images, you might want to do the resizing yourself in order to make sure the subject is fully in the picture.
- #86 I was never able to get the C++ script to work, but Python is here to rescue again. Now I don’t want to go through the retrain.py script line by line since it’s around 1000 lines, but in general, it downloads inception, pops off the last layer and retrains it with the images in our directory based on the folder structure.
- #91 A bottleneck is an “informal term” referring to the output of the previous layer. Since this process will want to refer to different images many times we want to cache these values. The output above is from running the retrain functionality with the bottlenecks already calculated so it was fast.
- #92 Tensorflow will split your data three ways - training, validation and testing. 70:20:10 - Training is the data you tune your model on, testing is the data your model never sees as a final set to test accuracy. - Validation is used to avoid overfitting - making sure improvements in training accuracy actually appear in an unseen dataset. - Cross entropy is your loss metric - it’s the metric the model is trying to minimize, rather than focusing directly on accuracy. That’s why it’s not a percentage.
- #96 AFTER: There is a label_image application written in C++, but I had no luck with that one
- #97 until with tf.Session()
- #105 And that’s the end, hopefully that was some food for thought for everyone