Quality and Testing of
Artificial Intelligence
Algorithms
5/3/2017
Sam Putnam, Enterprise Deep Learning, LLC
www.EnterpriseDeepLearning.com
Want to learn more?
I am teaching the Deploying Deep
Learning Track at Deep Learning Conf®
Tickets are available now:
www.DeepLearningConf.com
Topics
Machine Learning Algorithms in Production
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Spam Classification
Activity - Introductions
Recommendation Systems
Neural Networks
Decision Tree Classifiers
5/3/2017
Topics
Deploying Machine Learning Algorithms
Sam PutnamQuality and Testing of Artificial Intelligence Algorithms
Best Practices
Activity - Kaggle
Best Practices
Questions & Terminology
Packages Exist
5/18/2017
Topics
Algorithms in the Media
Sam PutnamQuality and Testing of Artificial Intelligence Algorithms
Deploying Machine Learning Algorithms
Activity - Combating Bias In Your Business’ Algorithms
Success Stories
Challenges/Pushback
AI-First
5/3/2017
Topics
Enterprise Risk Mitigation Framework
Sam PutnamQuality and Testing of Artificial Intelligence Algorithms
Evaluating Vendors
Activity - Workshop Close
Public and Private Data Sources
Your Ecosystem
Artificial Intelligence Today
5/3/2017
Part 1 of 4
Machine Learning Algorithms in Production
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017
http://benanne.github.io/2014/08/05/spotify-cnns.html
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
Activity - Name, Short
Professional Bio, Interest in
this Workshop
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017
https://www.youtube.com/watch?v=JHQ0Qv0l54U
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017
https://www.youtube.com/watch?v=6YJRujHRbb8
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017
http://deeplearning.lipingyang.org/2016/11/16/how-neural-networks-recognize-a-dog-in-a-photo/
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Machine Learning Algorithms in Production
5/3/2017
https://backchannel.com/inside-facebooks-ai-machine-7a869b922ea7
Part 2 of 4
Deploying Machine Learning Algorithms
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
Activity
Self-paced
1) Run Code
https://www.kaggle.com/sharmasanthosh/exploratory-
study-on-ml-algorithms
2) Collaborate with Neighbor on Questions
or
3) Ask Me
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Deploying Machine Learning Algorithms
5/3/2017
http://p.migdal.pl/2017/04/30/teaching-deep-learning.html
Part 3 of 4
Algorithms in the Media
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017
https://www.dailydot.com/debug/google-voice-recognition-gender-bias/
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017
http://www.businessinsider.com/predictive-policing-discriminatory-police-crime-2016-10
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017
http://www.independent.co.uk/life-style/gadgets-and-tech/news/googles-algorithm-shows-prestigious-job-ads-to-men-but-not-to-
women-10372166.html
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017
http://dilbert.com/strip/2013-02-02
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
Activity
1) What part of your business could be
affected by algorithms?
2) How could these algorithms be biased?
3) How could you mitigate this?
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017
http://www.economist.com/news/business/21720675-firm-using-algorithm-designed-cern-laboratory-how-germanys-otto-uses
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017http://www.newyorker.com/magazine/2017/04/03/ai-versus-md
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Algorithms in the Media
5/3/2017
https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awakening.html?_r=0
Part 4 of 4
Enterprise Risk Mitigation Framework
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
5/3/2017
https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/
can’t find original source. know it? email me - sam@EDeepLearning.com - I’ll add it
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
5/3/2017
https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/https://xkcd.com/1425/
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
5/3/2017
https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/
https://play.google.com/store/apps/details?id=com.google.android.googlequicksearchbox&hl=en
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
Activity
1) What did you learn?
2) What did you wish you could have learned?
3) What are you going to bring back to your
career?
5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
5/3/2017
https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/
https://twitter.com/fchollet/status/792960796133163008
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
5/3/2017https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/
https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
5/3/2017
https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/
can’t find original source. know it? email me - sam@EDeepLearning.com - I’ll add it
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
1. Search
‘Deep Learning’
2. Listen 5/3/2017
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Enterprise Risk Mitigation Framework
Artificial Intelligence
5/3/2017
https://blog.ycombinator.com/how-to-get-into-natural-language-processing/
Topics
Machine Learning Algorithms in Production
Deploying Machine Learning Algorithms
Algorithms in the Media
Enterprise Risk Mitigation Framework
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
5/3/2017
Thank you
Always looking for new members & new locations in Cambridge, US or
NYC & new speakers to present on Deep Learning in Production topics
Thank you to the Machine Learning Society and Anna Royzman
Quality and Testing of Artificial Intelligence Algorithms Sam Putnam
Thank you to Google, Amazon, Facebook, YC, DARPA, Others who have
published diagrams and photos. Slides are for today only. 5/3/2017
https://www.slideshare.net/anirudhkoul/squeezing-deep-learning-into-mobile-phones/77

Quality and Testing of AI Algorithms - Enterprise Deep Learning

  • 1.
    Quality and Testingof Artificial Intelligence Algorithms 5/3/2017 Sam Putnam, Enterprise Deep Learning, LLC www.EnterpriseDeepLearning.com Want to learn more? I am teaching the Deploying Deep Learning Track at Deep Learning Conf® Tickets are available now: www.DeepLearningConf.com
  • 2.
    Topics Machine Learning Algorithmsin Production Quality and Testing of Artificial Intelligence Algorithms Sam Putnam Spam Classification Activity - Introductions Recommendation Systems Neural Networks Decision Tree Classifiers 5/3/2017
  • 3.
    Topics Deploying Machine LearningAlgorithms Sam PutnamQuality and Testing of Artificial Intelligence Algorithms Best Practices Activity - Kaggle Best Practices Questions & Terminology Packages Exist 5/18/2017
  • 4.
    Topics Algorithms in theMedia Sam PutnamQuality and Testing of Artificial Intelligence Algorithms Deploying Machine Learning Algorithms Activity - Combating Bias In Your Business’ Algorithms Success Stories Challenges/Pushback AI-First 5/3/2017
  • 5.
    Topics Enterprise Risk MitigationFramework Sam PutnamQuality and Testing of Artificial Intelligence Algorithms Evaluating Vendors Activity - Workshop Close Public and Private Data Sources Your Ecosystem Artificial Intelligence Today 5/3/2017
  • 6.
    Part 1 of4 Machine Learning Algorithms in Production Quality and Testing of Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017
  • 7.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017 http://benanne.github.io/2014/08/05/spotify-cnns.html
  • 8.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017
  • 9.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production Activity - Name, Short Professional Bio, Interest in this Workshop 5/3/2017
  • 10.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
  • 11.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017 https://www.youtube.com/watch?v=JHQ0Qv0l54U
  • 12.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017 https://www.youtube.com/watch?v=6YJRujHRbb8
  • 13.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017 http://deeplearning.lipingyang.org/2016/11/16/how-neural-networks-recognize-a-dog-in-a-photo/
  • 14.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Machine Learning Algorithms in Production 5/3/2017 https://backchannel.com/inside-facebooks-ai-machine-7a869b922ea7
  • 15.
    Part 2 of4 Deploying Machine Learning Algorithms Quality and Testing of Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017
  • 16.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017 http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 17.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 18.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017 http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 19.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017 http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 20.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017 http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 21.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms Activity Self-paced 1) Run Code https://www.kaggle.com/sharmasanthosh/exploratory- study-on-ml-algorithms 2) Collaborate with Neighbor on Questions or 3) Ask Me 5/3/2017
  • 22.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017 http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 23.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017 http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 24.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
  • 25.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Deploying Machine Learning Algorithms 5/3/2017 http://p.migdal.pl/2017/04/30/teaching-deep-learning.html
  • 26.
    Part 3 of4 Algorithms in the Media Quality and Testing of Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017
  • 27.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017 https://www.dailydot.com/debug/google-voice-recognition-gender-bias/
  • 28.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017 http://www.businessinsider.com/predictive-policing-discriminatory-police-crime-2016-10
  • 29.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017 http://www.independent.co.uk/life-style/gadgets-and-tech/news/googles-algorithm-shows-prestigious-job-ads-to-men-but-not-to- women-10372166.html
  • 30.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017 http://dilbert.com/strip/2013-02-02
  • 31.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media Activity 1) What part of your business could be affected by algorithms? 2) How could these algorithms be biased? 3) How could you mitigate this? 5/3/2017
  • 32.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017 http://www.economist.com/news/business/21720675-firm-using-algorithm-designed-cern-laboratory-how-germanys-otto-uses
  • 33.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017http://www.newyorker.com/magazine/2017/04/03/ai-versus-md
  • 34.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Algorithms in the Media 5/3/2017 https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awakening.html?_r=0
  • 35.
    Part 4 of4 Enterprise Risk Mitigation Framework Quality and Testing of Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 5/3/2017
  • 36.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 5/3/2017 https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/ can’t find original source. know it? email me - sam@EDeepLearning.com - I’ll add it
  • 37.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 5/3/2017 https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/https://xkcd.com/1425/
  • 38.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 5/3/2017 https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/ https://play.google.com/store/apps/details?id=com.google.android.googlequicksearchbox&hl=en
  • 39.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework Activity 1) What did you learn? 2) What did you wish you could have learned? 3) What are you going to bring back to your career? 5/3/2017
  • 40.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 5/3/2017 https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/ https://twitter.com/fchollet/status/792960796133163008
  • 41.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 5/3/2017https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/ https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721
  • 42.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 5/3/2017 https://blog.ycombinator.com/how-to-know-when-products-actually-use-ai/ can’t find original source. know it? email me - sam@EDeepLearning.com - I’ll add it
  • 43.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework 1. Search ‘Deep Learning’ 2. Listen 5/3/2017
  • 44.
    Quality and Testingof Artificial Intelligence Algorithms Sam Putnam Enterprise Risk Mitigation Framework Artificial Intelligence 5/3/2017 https://blog.ycombinator.com/how-to-get-into-natural-language-processing/
  • 45.
    Topics Machine Learning Algorithmsin Production Deploying Machine Learning Algorithms Algorithms in the Media Enterprise Risk Mitigation Framework Quality and Testing of Artificial Intelligence Algorithms Sam Putnam 5/3/2017
  • 46.
    Thank you Always lookingfor new members & new locations in Cambridge, US or NYC & new speakers to present on Deep Learning in Production topics Thank you to the Machine Learning Society and Anna Royzman Quality and Testing of Artificial Intelligence Algorithms Sam Putnam Thank you to Google, Amazon, Facebook, YC, DARPA, Others who have published diagrams and photos. Slides are for today only. 5/3/2017 https://www.slideshare.net/anirudhkoul/squeezing-deep-learning-into-mobile-phones/77