Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Demystifying machine learning using lime

1,387 views

Published on

Slides of my Pycon 2017 short talk "Demystifying machine learning using lime"

Jupyter Notebook with code in https://github.com/albahnsen/Talk_Demystifying_Machine_Learning

Published in: Data & Analytics
  • Hello! High Quality And Affordable Essays For You. Starting at $4.99 per page - Check our website! https://vk.cc/82gJD2
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Demystifying machine learning using lime

  1. 1. DEMYSTIFYING MACHINE LEARNING Alejandro Correa Bahnsen 1
  2. 2. BLACK BOX MODELS Machine learning models are often dismissed on the grounds of lack of interpretability. When using advanced models it is nearly impossible to understand how a model is making a prediction. 2
  3. 3. MNIST - ACCU VS # PARAMS Notebook to create the plot 3
  4. 4. LIME stands for Local Interpretable Model- agnostic Explanations, and its objective is to explain the result from any classifier so that a human can understand individual predictions LIME 4
  5. 5. LIME An interpretable representation is a point in a space whose dimensions can be interpreted by a human. LIME frames the search for an interpretable explanation as an optimization problem. Given a set G of potentially interpretable models, we need a measure L(f,g,x) of how poorly the interpretable model g∈∈G approximates the original model f for point x this is the loss function. We also need some measure Ω(g) of the complexity of the model (e.g. the depth of a decision tree). We then pick a model which minimizes both of these ξ(x) = argmin g∈∈G L(f,g,x)+Ω(g) 5
  6. 6. LIME 6
  7. 7. LIME EXAMPLE URL PHISHING DETECTION 7
  8. 8. URL PHISHING CLASSIFIER Objective: Evaluate phishing probability using only the web site URL 8
  9. 9. URL PHISHING CLASSIFIER Train a random forest 9
  10. 10. LIME EXAMPLE Fit lime explainer Explain an instance 10
  11. 11. LIME EXAMPLE Example Phishing URL Phishing probability 1.0 Url = http://login.paypal.com.convexcentral.com/Update/ab770f624342b07b71e56c1bae5d9bcb/ 11
  12. 12. LIME EXAMPLE Example Phishing URL Phishing probability 0.0283 Url = ...http://www.redeyechicago.com/entertainment/tv/redeye-banshee-ivana-mili 12
  13. 13. THANK YOU FULL NOTEBOOK IN HTTPS://GITHUB.COM/ALBAHNSEN/TALK _DEMYSTIFYING_MACHINE_LEARNING 13

×