Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BigML Fall 2017 Release

324 views

Published on

Our fall 2017 release presents operating thresholds, available from the BigML Dashboard, API, and WhizzML for automation. The ability to set an operating threshold is a key feature that allows you to fine tune the performance of your classification models, and lets you be more or less aggressive when predicting a class for an instance. Applying operating thresholds is especially useful in domains like fraud detection or medical diagnosis, where the consequences of some classifications may have prohibitive costs associated. Additionally, BigML is bringing organizations, a convenient collaborative space where several customers can work on the same projects, within the same Dashboard, at different levels of privileges. Organizations break down silos and make it easy and efficient for any company to adopt Machine Learning across their entire corporate structure.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

BigML Fall 2017 Release

  1. 1. Introducing Operating Thresholds and Organizations BigML Fall 2017 Release
  2. 2. BigML, Inc 2BigML Release Webinar: Operating Thresholds and Organizations BigML Release CHARLES PARKER, PH.D. - VP of Machine Learning Algorithms Please enter questions into chat box – We will answer some via chat and others at the end of the session https://bigml.com/releases/fall-2017 ATAKAN CETINSOY - VP of Predictive Applications Resources Moderator Speaker Contact support@bigml.com Twitter @bigmlcom Questions
  3. 3. BigML, Inc 3BigML Release Webinar: Operating Thresholds and Organizations • You have a model! What will you do with it? • Make Predictions • About what? And what happens afterwards? • You probably need an operating threshold • Share it • With whom? • Use organizations Actually using a model
  4. 4. BigML, Inc 4BigML Release Webinar: Operating Thresholds and Organizations • The model has no idea what you want! • It’s optimizing its own version of loss • There are some ways to tell the model to learn differently • Class weights / Balance objective • Weight column • A lot of times, you can apply your preferences after learning, but only if you know what they are • You can discover them using an evaluation The Dirty Little Secret of ML
  5. 5. BigML, Inc 5BigML Release Webinar: Operating Thresholds and Organizations Evaluation Curves
  6. 6. BigML, Inc 6BigML Release Webinar: Operating Thresholds and Organizations • The evaluation tool gives you the opportunity to see the effect of setting different thresholds for a particular class on your data • If the probability of this class is over the threshold, BigML predicts it • Else BigML chooses the next-highest probability class The Threshold Slider
  7. 7. BigML, Inc 7BigML Release Webinar: Operating Thresholds and Organizations • Suppose the costs of false positives and false negatives are different • Medical Diagnosis: • The positive class is rare, and so will not be predicted often • The positive class is very important! • Use the evaluation tool to set the tradeoff between the error types Case 1: Medical Diagnosis
  8. 8. BigML, Inc 8BigML Release Webinar: Operating Thresholds and Organizations • Sometimes you know precisely the cost of making a classification • Predicting failure for a machine: • Each positive prediction requires you to send a tech out into the field ($) • Each false negative costs some amount of downtime ($$$), plus the cost of the tech • True negatives cost nothing • Solution: Use WhizzML on the evaluation • Coming soon to an interface near you! Case 1-1: Failure Prediction
  9. 9. BigML, Inc 9BigML Release Webinar: Operating Thresholds and Organizations • Predict: When a customer is going to leave a subscription service; take some action to retain them if they are • Sometimes you have a constraint on the number of times you can apply a class • In churn prediction, you want to call the people most likely to churn • But you only have so much money to pay for it • Set the threshold according to your budget; use the evaluation tool to estimate performance Case 2: Churn Prediction
  10. 10. BigML, Inc 10BigML Release Webinar: Operating Thresholds and Organizations • Sometimes you have a fixed constraint, but the input domain changes • Suppose you have a lending operation where your funding allows you to approve half of the applications that come in • What happens if I suddenly start getting twice as many applications? • Use the original evaluation to reset the threshold (assuming the same application quality) Case 2-2: Credit Default
  11. 11. BigML, Inc 11BigML Release Webinar: Operating Thresholds and Organizations • Now that you have your threshold, what do you do? • BigML makes predicting with a threshold BigML easy • Thresholds available for predictions, batch predictions, through the BigML API, and WhizzML for automation Now What?
  12. 12. BigML, Inc 12BigML Release Webinar: Operating Thresholds and Organizations • Some models have support for confidence, which is an instance-dependent lower bound on the probability of the model • If the model supports confidence, you can threshold on that as well Using Confidence
  13. 13. BigML, Inc 13BigML Release Webinar: Operating Thresholds and Organizations • Machine learning often involves a lot of duplicated work • Preprocessing data • Updating models • It can be ridiculous to have this duplicated in multiple places in the same company / working group • BigML organizations is a fix for this problem Organizations
  14. 14. BigML, Inc 14BigML Release Webinar: Operating Thresholds and Organizations • Create an organization • Organizations are separate from the user’s individual account • Other users can be invited to the organization Creating an Organization
  15. 15. BigML, Inc 15BigML Release Webinar: Operating Thresholds and Organizations Managing Users • Each user in the organization has a certain role, which controls their ability to access and create resources
  16. 16. BigML, Inc 16BigML Release Webinar: Operating Thresholds and Organizations • Each project can have access settings per user (read, create, etc.) • The organization can have many projects Assigning Permissions
  17. 17. BigML, Inc 17BigML Release Webinar: Operating Thresholds and Organizations More Info https://bigml.com/releases/fall-2017
  18. 18. Questions? @bigmlcom support@bigml.com

×