Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Executing Deep Learning Strategies Masterclass Preview - Enterprise Deep Learning

457 views

Published on

Day 1 - New York

Published in: Software
  • Be the first to comment

  • Be the first to like this

Executing Deep Learning Strategies Masterclass Preview - Enterprise Deep Learning

  1. 1. Executing Deep Learning Strategies Sam Putnam, Enterprise Deep Learning, LLC July 26, 2017 Day 1
  2. 2. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Morning: Introduction to deep learning - High-level overview of why enterprises are using deep learning 9:00 – 10:00 Introduction to machine learning and its applications 10:00 - 11:00 Introduction to deep learning and how it is currently being used by enterprises 11:00 - 12:00 Question and answer session with each student on deep learning's present or future role in their business 12:00-1:00 Lunch Afternoon: Overview of deep learning strategies
 1:00-2:00 Awareness building - educating specific groups within your enterprise on the maturity of the technology 2:00-3:00 Specialized to particular tasks - bespoke, custom, made-to-order solutions 3:00-4:00 Using deep learning across teams - capturing and re-using insights, running a deep learning-first enterprise 4:00-5:00 Question and answer session with each student on what their business would need out of a deep learning strategy July 26 2017
  3. 3. Part 1 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning July 26 2017 Introduction to Machine Learning and its Applications
  4. 4. Part 2 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Introduction to Deep Learning and How It Is Being Used By Enterprises July 26 2017
  5. 5. Part 3 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Question and Answer Session on Deep Learning’s Current or Future Role In Your Business July 26 2017
  6. 6. Part 4 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Awareness Building - Educating Specific Groups Within Your Enterprise on the Maturity of the Technology July 26 2017
  7. 7. Part 5 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions July 26 2017
  8. 8. Part 6of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Using Deep Learning Across Teams - Capturing and Re-using Insights, Running a Deep-Learning First Enterprise July 26 2017
  9. 9. Part 7 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Question and Answer Session on What Your Business Would Need Out Of A Deep Learning Strategy July 26 2017
  10. 10. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Anomaly Detection - detecting an insider trade, flagging a keyword - detecting a fraudulent credit card transaction - detect money laundering Machine Learning Applications Speech Recognition - detecting specific words for translation systems - recognizing your voice, for biometric systems - identify whale sounds, so ships do not hit them - detect bird species from a bird call July 26 2017
  11. 11. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Face Detection Preview - Slide Available at deeplearningconf.com Machine Learning Applications Facial Recognition Preview - Slide Available at deeplearningconf.com July 26 2017
  12. 12. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Image Classification - tagging images to create a searchable database - monitoring a stream of images - augmenting security, helping those who label data - segment objects, crop photos Machine Learning Applications Topic Modeling - automatic tagging of news articles, written content - detecting a fraudulent transaction, July 26 2017
  13. 13. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Anti-Spam - detecting (lazy) malware, use system logs/packets - detect spam in your email, want few false positives - detecting spam blogs, SEO aggressors Machine Learning Applications Genetics - use clustering to find disease-predicting genes July 26 2017
  14. 14. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Search Preview - Slide Available at deeplearningconf.com Machine Learning Applications Ads Preview - Slide Available at deeplearningconf.com July 26 2017
  15. 15. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Translation - translate another language to native language - translate obscure text/writing into readable English Machine Learning Applications Language understanding - understand intents/slots from a query - understand text, forward to appropriate recipient - branch to different department, issue tracking - summarize long documents July 26 2017
  16. 16. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Video compression Preview - Slide Available at deeplearningconf.com Machine Learning Applications Weather forecasting Preview - Slide Available at deeplearningconf.com Market segmentation Preview - Slide Available at deeplearningconf.com July 26 2017
  17. 17. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Price premiums - forecast the lifetime cost of an insurance customer Machine Learning Applications Trade stocks and derivatives - buy, clean, and backtest data with algorithms - extract true patterns and detect buy signals Predict risk of investment - use key performance indicators - test and mine valuation models July 26 2017
  18. 18. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Forecast housing/auction sale prices Preview - Slide Available at deeplearningconf.com Machine Learning Applications Predict ratings Preview - Slide Available at deeplearningconf.com July 26 2017
  19. 19. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Recognize characters Preview - Slide Available at deeplearningconf.com Machine Learning Applications Self-driving cars Preview - Slide Available at deeplearningconf.com July 26 2017
  20. 20. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Factory equipment maintenance Preview - Slide Available at deeplearningconf.com Machine Learning Applications Wait times Preview - Slide Available at deeplearningconf.com July 26 2017
  21. 21. Other forms of fraud Preview - Slide Available at deeplearningconf.com Sam PutnamExecuting Deep Learning Strategies @edeeplearning In-person market segmentation Preview - Slide Available at deeplearningconf.com Machine Learning Applications Chemical drug discovery Preview - Slide Available at deeplearningconf.com July 26 2017
  22. 22. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Music recommendations Preview - Slide Available at deeplearningconf.com Machine Learning Applications News or product recommendations Preview - Slide Available at deeplearningconf.com More recommendations Preview - Slide Available at deeplearningconf.com July 26 2017
  23. 23. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Employee permissions - granting or revoking access to specific applications - employee attrition Machine Learning Applications Medical event premonition - analyze doctors’ notes to predict heart failure - predict emergency room admissions - predict premature births July 26 2017
  24. 24. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Call for help Preview - Slide Available at deeplearningconf.com Machine Learning Applications Video Event Detection/Anomaly Event detection Preview - Slide Available at deeplearningconf.com July 26 2017
  25. 25. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Design suggestions - recommend layouts that match color palettes - colorize black and white images using context - generate variations of drawing created by user - create a full photo from a sketch Machine Learning Applications Identify songs - hear a few seconds of a song and give the title July 26 2017
  26. 26. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Generate handwriting - create personalized experiences Machine Learning Applications Generating Text - automatically caption images - generate new ads from previous ad clicks/social - Fill in missing parts of a legal document - Generate coherent arguments from legal documents July 26 2017
  27. 27. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Synthesize sound Preview - Slide Available at deeplearningconf.com Machine Learning Applications Generating & Organizing Music Preview - Slide Available at deeplearningconf.com July 26 2017
  28. 28. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Question answering - interact with a chatbot to inquire about an item - ask your phone questions, receive useful answers - feature suggestions, gradually expose user via chat Machine Learning Applications Understand emotion - See images from a video stream and read emotions - Identify email or chat messages that are sales leads - Triage users who need special care or attention - Personality detection and compatibility July 26 2017
  29. 29. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Individual detection - enable actions for specific people in a seat in a car - personalize/identify user of a family internet account Machine Learning Applications Predict rising/trending stars - Mine tags in a geographic area for keywords - Predict if a product launch will be successful - Identify credibility of a source or thought leader July 26 2017
  30. 30. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Flag noteworthy news Preview - Slide Available at deeplearningconf.com Machine Learning Applications Authentication Preview - Slide Available at deeplearningconf.com July 26 2017
  31. 31. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Security - predictive policing using crime data - imaging and video systems for airport security - predict psychopathy from internet usage Machine Learning Applications Energy - estimating demand requirements, load balancing - initiating iOT devices to turn on at low peak times July 26 2017
  32. 32. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Predict bad loans - predict refinancers and defaulters - analyze credit risk and automate loan processing - conversely reward those most likely to pay back Machine Learning Applications Combinations of Artistic styles - combine a famous painting with a camera photo - modify a live video stream with artistic style July 26 2017
  33. 33. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Detect new market segmentations Preview - Slide Available at deeplearningconf.com Machine Learning Applications Text to speech and speech to text Preview - Slide Available at deeplearningconf.com July 26 2017
  34. 34. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Machine Learning Applications Pretty much anything that a normal person can do in <1 sec, we can now automate with AI. – Andrew Ng (@AndrewYNg), 19 October 2016 (with enough developers, even physical motion) July 26 2017
  35. 35. Sam PutnamExecuting Deep Learning Strategies @edeeplearning What is Machine Learning? July 26 2017 A: It Is Teaching a Machine B: It’s The Engine Behind Artificial Intelligence (AI) You Might Believe: C: It’s What Makes Google Still Relevant and What Makes New Startups Interesting Enough To Compete (This is a cop out Answer)
  36. 36. Sam PutnamExecuting Deep Learning Strategies @edeeplearning What is Machine Learning? A: It Is Teaching a Machine B: It’s The Engine Behind Artificial Intelligence (AI) You Might Believe: C: It’s What Makes Google Still Relevant and What Makes New Startups Interesting Enough To Compete (This is a cop out Answer) D: It’s writing code that allows a computer to mine vast quantities of data and make fast, intelligent decisions July 26 2017
  37. 37. Sam PutnamExecuting Deep Learning Strategies @edeeplearning What is Machine Learning? A: It Is basic intelligence that an analyst can do, but takes time B: It is finding simple patterns in large amounts of data that humans can’t physically process often sub-human or comparable performance that saves money Other Adequate answers: July 26 2017
  38. 38. Sam PutnamExecuting Deep Learning Strategies @edeeplearning What is Machine Learning? Preview - Slide Available at deeplearningconf.com July 26 2017
  39. 39. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Preview - Slide Available at deeplearningconf.com July 26 2017
  40. 40. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ - You could use only common sense, which you may have a lot of, if you have domain knowledge Preview - Slide Available at deeplearningconf.com July 26 2017
  41. 41. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ So, you pick a decision threshold July 26 2017
  42. 42. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ - You can classify data by only one feature, and it is scientific to look at one variable, but it’s very limiting! Preview - Slide Available at deeplearningconf.com July 26 2017
  43. 43. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ Preview - Slide Available at deeplearningconf.com July 26 2017
  44. 44. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ - You can classify data by only two features, and it is interpretable, but it’s still limiting! Preview - Slide Available at deeplearningconf.com July 26 2017
  45. 45. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ - Question: Why is it interpretable? - Answer: You can draw the decision boundary (see the larger green and smaller blue rectangles here), and therefore visualize the separation between your two classes. July 26 2017
  46. 46. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ - So, you could add more features. - Here, you have seven features, with the relationship between the features plotted. Look at the top row, what have you learned? Perhaps still nothing more than the fact that houses in SF (green) are at higher elevations than houses in NY (blue). But, In most of the plots, the boundaries that delineate the classes are not immediately clear. Machine Learning solves this problem for you. Preview - Slide Available at deeplearningconf.com July 26 2017
  47. 47. Sam PutnamExecuting Deep Learning Strategies @edeeplearning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/ Preview - Slide Available at deeplearningconf.com July 26 2017
  48. 48. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Preview - Slide Available at deeplearningconf.com July 26 2017
  49. 49. Sam PutnamExecuting Deep Learning Strategies @edeeplearning What is Deep Learning? Automate Machine Learning Automated feature selection Look at larger quantities of data Improve accuracy even more in industries that can benefit from small to medium improvements July 26 2017
  50. 50. Part 2 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Introduction to Deep Learning and How It Is Being Used By Enterprises July 26 2017
  51. 51. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Compliance - Bank Secrecy Act Anti-Money Laundering BSA AML - cluster suspicious transactions - improvement on rule-based alert(s), which do not evolve https://skymind.ai/case-studies/bsa-aml - can have basic machine learning system that adds rules July 26 2017
  52. 52. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises SIM Box Fraud https://skymind.ai/case-studies/orange Preview - Slide Available at deeplearningconf.com July 26 2017
  53. 53. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises https://skymind.ai/case-studies/canonical SysAdmin Monitoring Preview - Slide Available at deeplearningconf.com July 26 2017
  54. 54. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises SysAdmin Monitoring Process https://skymind.ai/case-studies/canonical Preview - Slide Available at deeplearningconf.com July 26 2017
  55. 55. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Fraud Monitoring https://skymind.ai/case-studies/finance - rule based systems used because tens to hundreds of authorizations per second are sent to be vetted - instead of sampling fraud instances and losing information, collect and learn from all of the information July 26 2017
  56. 56. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - roll your own system, rather than relying on legacy, catch-all analytics systems - adaptively prioritize high probability cases of frauds in ways frozen decision tree models do not Deep Learning and How It Is Being Used By Enterprises Fraud Monitoring https://skymind.ai/case-studies/finance - finance is already open source - Linux, Hadoop/ Spark July 26 2017
  57. 57. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises https://skymind.ai/case-studies/image Image classification for online experiences, e-commerce, and security Preview - Slide Available at deeplearningconf.com July 26 2017
  58. 58. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises https://skymind.ai/case-studies/image Image classification for security and enforcement - detect logos in images for copyright infringement - monitor video streams and detect weapons under clothing - automobile industry is a user of deep learning, detect scratches on vehicles as they go down assembly line July 26 2017
  59. 59. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises https://skymind.ai/case-studies/image Image classification for security and enforcement Preview - Slide Available at deeplearningconf.com July 26 2017
  60. 60. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises https://skymind.ai/commerce Recommender Systems - deal with mix of browsing behavior metadata, support chat history, and transaction history - analyze information about pages viewed, text in descriptions, product images, type of music played in store - constantly refine clustering algos July 26 2017
  61. 61. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Intro to Neural Networks (Part 1: Data and Architecture) Keywords: Supervised, Regression, Classification, Artificial Neural Networks, Hidden Layers, Weights https://www.youtube.com/watch?v=bxe2T-V8XRs Preview - Slide Available at deeplearningconf.com July 26 2017
  62. 62. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Intro to Neural Networks (Part 2: Forward Propogation) Keywords: Hyperparameters, Activation Function, Forward Propagation, Hidden Layers https://www.youtube.com/watch?v=UJwK6jAStmg Preview - Slide Available at deeplearningconf.com July 26 2017
  63. 63. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Intro to Neural Networks (Part 3: Gradient Descent) Keywords: Cost Function, Training, Dimensionality, Brute Force, Gradient Descent, Non-Convex, Stochastic Gradient Descent https://www.youtube.com/watch?v=5u0jaA3qAGk Preview - Slide Available at deeplearningconf.com July 26 2017
  64. 64. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Intro to Neural Networks (Part 4: Backpropagation) Keywords: Chain Rule, Activation, Backprogagating, Batch Gradient Descent, Deep Neural Network, https://www.youtube.com/watch?v=GlcnxUlrtek Preview - Slide Available at deeplearningconf.com July 26 2017
  65. 65. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Intro to Neural Networks (Part 5: Numerical Gradient Checking) Keywords: The Definition of the Derivative, Vectors, Perturb Weights, Norm https://www.youtube.com/watch?v=pHMzNW8Agq4 Preview - Slide Available at deeplearningconf.com July 26 2017
  66. 66. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Intro to Neural Networks (Part 6: Training) Keywords: Convergence, Local Minima, Optimization, BFGS, Overfitting, https://www.youtube.com/watch?v=bxe2T-V8XRs Preview - Slide Available at deeplearningconf.com July 26 2017
  67. 67. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep Learning and How It Is Being Used By Enterprises Intro to Neural Networks (Part 7: Overfitting, Testing, and Regularization) Keywords: Signal vs. Noise, Training Data, Testing Data, Regularization https://www.youtube.com/watch?v=bxe2T-V8XRs Preview - Slide Available at deeplearningconf.com July 26 2017
  68. 68. Part 3 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Question and Answer Session on Deep Learning’s Current or Future Role In Your Business July 26 2017
  69. 69. Sam PutnamExecuting Deep Learning Strategies @edeeplearning 1) What is your name, again, and what is your business? 2) What do you see as deep learning’s current or future role in your business? 3) What questions do you have about deep learning in production that we can we talk through? July 26 2017
  70. 70. Part 4 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Awareness Building - Educating Specific Groups Within Your Enterprise on the Maturity of the Technology July 26 2017
  71. 71. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - No decision is the default for machine learning systems - that is Ok If the current system is A, then the team would be unlikely to switch to B. If the current system is B, then the team would be unlikely to switch to A. This seems in conflict with rational behavior: however, predictions of changing metrics may or may not pan out, and thus there is a large risk involved with either change. Start with Expectations and Metrics http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf July 26 2017
  72. 72. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - You may have a number of project requirements and metrics, but you start with one machine learning objective function - If you optimize for number of clicks, you are likely to see the time spent increase. So, keep it simple and don’t think too hard about balancing different metrics when you can still easily increase all the metrics. Have 1 Objective Only July 26 2017
  73. 73. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Launch first, get data and do machine learning after Preview - Slide Available at deeplearningconf.com July 26 2017
  74. 74. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Do best to capture all information Preview - Slide Available at deeplearningconf.com July 26 2017
  75. 75. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Keep it simple Preview - Slide Available at deeplearningconf.com July 26 2017
  76. 76. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Have specific people own a feature/column and document how it is populated and what it is - Although many feature columns have descriptive names, it's good to have a more detailed description of what the feature is, where it came from, and how it is expected to help. Segment ownership of the project to data engineering and data science July 26 2017
  77. 77. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Prioritize the freshness requirement of your model, if it generates a majority of your revenue and you've seen a major performance hit when its unwatched, watch it more often - Ad systems receive new ads each day, so generally must update daily Freshness is paramount July 26 2017
  78. 78. Part 5 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions July 26 2017
  79. 79. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - 5.5 million face videos from 70 countries Preview - Slide Available at deeplearningconf.com July 26 2017
  80. 80. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Quantify gaze fixation duration, but also discrete emotions (happy, sad, perplexed) - Partner company, HireVue uses tech to rank video interviews - Recently started to use voice data as well, another modality, conversational commerce, healthcare proven by other companies July 26 2017
  81. 81. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Use temporal model, expressions unfold over time Preview - Slide Available at deeplearningconf.com July 26 2017
  82. 82. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Using Convolutional Neural Nets, training on identical images with different lighting for robustness to daylight/nighttime - All data goes to Amazon S3 - Certified labelers spit into training and validation, if agree - Also use active machine learning, so machine in the loop, humans decide outliers July 26 2017
  83. 83. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Typically train on 200 examples, run experiments every night Preview - Slide Available at deeplearningconf.com July 26 2017
  84. 84. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Think car on autopilot, in seconds needs to know if you’re awake or not to take control - Think application that reads your emotions as you view a billboard, car iOT Preview - Slide Available at deeplearningconf.com July 26 2017
  85. 85. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Matters because techniques are very applicable to audio, using analog detectors to get audio waveform out, extracting the signature of a particle, as in speech Preview - Slide Available at deeplearningconf.com July 26 2017
  86. 86. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Cofounder recorded himself 24/7 for two weeks, uploaded the data every day - Speech to Text didn’t work well enough, average microphone caused high word error rate - Google had worked on this problem in the past, turning political speeches to text, but didn’t work July 26 2017
  87. 87. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Google went back to text to speech - Deepgram folks tried to recognize features directly, started at 20% accuracy, went to 90% six months later - Think searching/indexing recorded business calls that would otherwise be useless low quality audioJuly 26 2017
  88. 88. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Not using text as an intermediary, building the index out of activations in the deep neural network - Similar to using phonemes as features, but doesn’t exactly learn same “phonemes” - Can force an ASR system to use phonemes to get slightly better performance in cases July 26 2017
  89. 89. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Same idea as CNNs, hierarchical representations, deeper is higher level, as in phonemes versus indiscernible sounds - Can train with labeled topics and part of speech to further restrict and control what the activations are - Can look at 2D FFT and recognize patterns, like images July 26 2017
  90. 90. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - In the tens of layers could go deeper with much more computation, but need to think where to put skips and 90% to 92% not that attractive at this time - Service is an API, send in query and get all mentions - Can get customized model built, needs 50 to 100 ten minute to one hour long files at least July 26 2017
  91. 91. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Own model runs on top of Theano, Caffe, and TensorFlow, can add 2D convolutional layers, another with batch norm, recurrent layers with batch norm, then dense layer that predicts the word Preview - Slide Available at deeplearningconf.com July 26 2017
  92. 92. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Three types of cybersecurity categories to consider: Malware, threat detection, and stream detection Preview - Slide Available at deeplearningconf.com July 26 2017
  93. 93. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Ground truth is really expensive, unlike the image and video data applications, where data is plentiful - To get the “bad” data, pay a provider for a threat database Preview - Slide Available at deeplearningconf.com July 26 2017
  94. 94. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Clients are antivirus companies and police organizations - Need an expert to tell you that is a threat, must reverse engineer it Preview - Slide Available at deeplearningconf.com July 26 2017
  95. 95. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Have a labeled database, so run an ensemble regression on the inputs, use scoring altos to see how serious the data is coming from the feed/stream - Also collect data with “honeypots” - vulnerable machines put out as bait, get info on attackers July 26 2017
  96. 96. - Have both host and network based data to look at, common to look at function names and regex for “bad” strings Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Common technique is to look at the file hash - the filename description, in large quantities, to classify a threat Preview - Slide Available at deeplearningconf.com July 26 2017
  97. 97. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Bad actors might add gibberish to a file, that can mislead tools Preview - Slide Available at deeplearningconf.com July 26 2017
  98. 98. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Plus, calls to 1000 domains, picks one or two to attahk, modifies domains using domain generation algos, which humans can easily see through, but not hundreds of thousands a day Preview - Slide Available at deeplearningconf.com July 26 2017
  99. 99. - Problem: adversaries follow conferences, two to three weeks see big change in variance in random domains Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - A regex can’t identify these pseudorandom domain names, so unsupervised learning is used Preview - Slide Available at deeplearningconf.com July 26 2017
  100. 100. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Develop predictive models that cover all major trading markets in the world, 30k stocks, forecast over 7 days, 14 days, and 30 days, seeing 60% to 70% accuracy four months in, so far Preview - Slide Available at deeplearningconf.com July 26 2017
  101. 101. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Macroeconomic effects - Something in Asia ripples through the US financial markets - Bringing technology to hedge fund quants as early adopters Preview - Slide Available at deeplearningconf.com July 26 2017
  102. 102. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Ultimately make accessible to investment managers - want capabilities exposed via natural language Preview - Slide Available at deeplearningconf.com July 26 2017
  103. 103. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Traditional model can’t represent graph-directed edges, monitoring a million pieces of data a day - Structured financial data and unstructured regulatory filings - 300k news articles a day also - topic modeling July 26 2017
  104. 104. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Cover 80k publicly traded companies, 15 million private companies, “track” 200 million people, current litigation, who investors are Preview - Slide Available at deeplearningconf.com July 26 2017
  105. 105. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Entity resolution, disambiguating between sea shells and Shell the company, record linkage links datasets together, connecting things based off of location, or mention of an iPhone, even when Apple not mentioned Preview - Slide Available at deeplearningconf.com July 26 2017
  106. 106. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Grab every patent of every US company, check if address matches other company filings, look at these bodies of evidence to prove a linkage - Automated information extraction a real challenge, use deep learning with TensorFlow for language understanding, and to extract similar entities July 26 2017
  107. 107. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Get better results using thousands of highly specialized models than one general AI model - First network picks out a concept/topic from the text, then a more sophisticated model determines that the document is about, say, a corporation. July 26 2017
  108. 108. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Given word embeddings, corporation is near M & A, strong relationship there, so can identify that original company is involved in M & A activity. - Think automated tone identification to augment committees of users who currently evaluate CFO’s body language and mannerisms July 26 2017
  109. 109. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions - Bottom line - deep learning at the onset for topic modeling and summarization, interpretable models at the user-facing end for the forecasting - Launched product, using an ensemble model so that is more “reverse engineerable”, retrain when you find you overweighted interest rates, but try to be hands free, 60% is good enough oner an entire portfolio July 26 2017
  110. 110. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Specialized to Particular Tasks - Bespoke, Custom, Made- to-Order Solutions Preview - Slide Available at deeplearningconf.com July 26 2017
  111. 111. Part 6of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Using Deep Learning Across Teams - Capturing and Re-using Insights, Running a Deep-Learning First Enterprise July 26 2017
  112. 112. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Use Analytics - Use analytics, choose simple features you don’t know you will need yet, track your current system with metrics http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf Preview - Slide Available at deeplearningconf.com July 26 2017
  113. 113. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Choose simple features so that you can be assured they are reaching your algorithm correctly when you have a mix of live and offline-collected features Create your pipeline Preview - Slide Available at deeplearningconf.com July 26 2017
  114. 114. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Have permission to manually inspect data to double check? Do it. - Have tests for examples in training and serving and check that the score is the same for an example Test, independently Preview - Slide Available at deeplearningconf.com July 26 2017
  115. 115. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - If data is coming from different services in your company, include a feature that specifies this Include most data Preview - Slide Available at deeplearningconf.com July 26 2017
  116. 116. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Use preexisting systems for preprocessing for your new system, a sender that has been blacklisted should be labeled such, not learned - Create a feature that encodes the heuristic, a relevance score can be encoded as a feature for a search system, just as a tax assessment can be encoded for a housing prices model Use what you have previously learned July 26 2017
  117. 117. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Do create composite heuristics that well represent your scoring function, for example, multiple the rating by the number of installs - Try breaking apart a composite heuristic of unrelated features, say one that sums installs and number of characters in an app description, and use each of those parts as features Cautiously use combined heuristics July 26 2017
  118. 118. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - If you have a significant difference between training performance and holdout performance, look to training the next iteration of your model using data that is from different days - Codify positional features, you will sidestep feedback loops that come from placing, for an example, an ad in the first position and seeing it get clicked and therefore weighted more heavily Check for data collection missteps and feedback loops July 26 2017
  119. 119. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Have higher regularization on features that correspond to more than one query, that way you can emphasis results that respond only to specific queries - Only allow features to have positive weights; therefore, only “good” features will be used, no feature will have a more negative impact than an unknown feature Make informed choices for parameters July 26 2017
  120. 120. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - The data bottleneck becomes apparent with unbalanced classes. If you sampling a tiny percentage, 0.01%, and you have few spam examples, you must uphold the sampling rate between classes. You can work with as few as 10k examples, sometimes fewer You don’t need all the data, if you have a ton July 26 2017
  121. 121. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Use a basic metric, and separate from production initially Preview - Slide Available at deeplearningconf.com July 26 2017
  122. 122. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Gradually decaying performance can indicate a table is not being refreshed, so a feature has significantly and/or older examples, so a refresh can improve performance vastly more than improving the model Even the giants fail, conspicuously in this case Preview - Slide Available at deeplearningconf.com July 26 2017
  123. 123. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Pick one proxy objective and stick with it at the onset - If your objective is increased, but you have chosen not to launch the product, reevaluate or pick a new objective that when increased will result in a launch Preview - Slide Available at deeplearningconf.com July 26 2017
  124. 124. Sam PutnamExecuting Deep Learning Strategies @edeeplearning 1. Was this ranked link clicked? 2. Was this ranked object downloaded? 3. Was this ranked object forwarded/replied to/emailed? 4. Was this ranked object rated? 5. Was this shown object marked as spam/pornography/ offensive? Choose direct objectives: Pick a simple objective July 26 2017
  125. 125. Sam PutnamExecuting Deep Learning Strategies @edeeplearning 1. Did the user visit the next day? 2. How long did the user visit the site? 3. What were the daily active users? . Avoid modeling indirect effects, these are metrics, to be used during A/B testing during launch, not objectives: Watch out for roundabout or indirect objectives July 26 2017
  126. 126. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Get as many metrics as possible, combine and recombine features to create new features, just like that embraced by Kaggle Iterate on features for your models Preview - Slide Available at deeplearningconf.com July 26 2017
  127. 127. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Use deep features after you have a baseline system using manually selected features, as a deep model can give a different solution the next time it is trained, so it needs to be tested more thoroughly Use deep features after you have developed a baseline Preview - Slide Available at deeplearningconf.com July 26 2017
  128. 128. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - It is expected that some of the labels will be wrong or cover different amounts of data, use a lot of these simple features anyway. You can use “regularization”, that is, add noise, to learn in a generalizable way, despite this. Use all the data you have, but nothing you wouldn’t have Preview - Slide Available at deeplearningconf.com July 26 2017
  129. 129. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Deep learning across the teams must be calculated Preview - Slide Available at deeplearningconf.com July 26 2017
  130. 130. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Diversity, relevance, and personalization are not as correlated with popularity, that is, clicks, as one might think Look at just your objective Preview - Slide Available at deeplearningconf.com July 26 2017
  131. 131. Sam PutnamExecuting Deep Learning Strategies @edeeplearning Look for new data sources once you plateau Preview - Slide Available at deeplearningconf.com July 26 2017
  132. 132. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - If you can use one programming language across training and serving, do it, it will allow you to share code and better confirm performance Simplify and do not cross models Preview - Slide Available at deeplearningconf.com July 26 2017
  133. 133. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Pick a number of features that represent the length, such as number of unique words, number of words, number of characters, number of pages, for example, let the system decide what is important. Pick multiple features, let the system learn Preview - Slide Available at deeplearningconf.com July 26 2017
  134. 134. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Engineering time is too valuable to spend time guessing at the importance of certain features as compared to others Spend engineering time engineering Preview - Slide Available at deeplearningconf.com July 26 2017
  135. 135. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Get rid of features that are not clean or interpretable as to how they were collected, or that cover a tiny percentage of the examples - At the same time, definitely keep those features that cover a tiny percentage of the examples but a large percentage of a certain class Clean up your features if you can July 26 2017
  136. 136. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - The number of feature weights you can learn, that is, the number of parameters, is roughly proportional to the amount of data you have - Yes, more data means more features, which tends to mean better performance. As a general rule, go two orders of magnitude down from number of examples to number of features Follow general rules July 26 2017
  137. 137. Sam PutnamExecuting Deep Learning Strategies @edeeplearning - Preprocess data using “discretizations" and “crosses”. - Discretizations create boundaries of histograms, pick a discrete integer to represent a grouping - With a large amount of data, use crosses to create new features that are unions/overlaps between 2 or more features Use transformations to augment data July 26 2017
  138. 138. Part 7 of 7 Sam PutnamExecuting Deep Learning Strategies @edeeplearning Question and Answer Session on What Your Business Would Need Out Of A Deep Learning Strategy July 26 2017
  139. 139. Sam PutnamExecuting Deep Learning Strategies @edeeplearning 1) What do you see as your company’s deep learning strategy? 2) What questions do you have about deep learning strategies that we can we talk through? 3) What is something that you are going to take away from this conference and apply to your business? July 26 2017
  140. 140. Thank you Sam Putnam Questions/Comments: Sam@EDeepLearning.com Thank you to Google and others who have published guidelines. Slides are for today only. Executing Deep Learning Strategies @edeeplearning July 26 2017

×