• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
509
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Online Random Forest in 10 Minutes
  • 2. Traditional Supervised Learning Algorithms ● ● ● ● ● Regression Random Forest Support Vector Machines Classification and Regression Tree (CART) etc
  • 3. Inputs ● Data Matrix (Regression) Predictand Predictor 1 Predictor 2 Predictor 3 Predictor 4 .56 Red .456 Male .589 .78 Green .654 Female .6654 .987 Blue .678 Female .789 .123 Blue .999 Male .543
  • 4. Inputs ● Data Matrix (Binary Classification) Predictand Predictor 1 Predictor 2 Predictor 3 Predictor 4 Yes Red .456 Male .589 No Green .654 Female .6654 Yes Blue .678 Female .789 No Blue .999 Male .543
  • 5. Inputs To Streaming Classification ● Observations now have an explicit arrival order. Predictand Predictor 1 Predictor 2 Predictor 3 Predictor 4 Time Yes Red .456 Male .589 Jan 1st 2011 No Green .654 Female .6654 Feb 4th 2012 Yes Blue .678 Female .789 Feb 5th 2013 No Blue .999 Male .543 July 4th
  • 6. Inputs To Streaming Classification ● New Observations can arrive at any time Predictand Predictor 1 Predictor 2 Predictor 3 Predictor 4 Time Yes Red .456 Male .589 Jan 1st 2011 No Green .654 Female .6654 Feb 4th 2012 Yes Blue .678 Female .789 Feb 5th 2013 No Blue .999 Male .543 July 4th 2013 Yes Red .456 Male .456 NOW
  • 7. Problems ● Do the important predictors change over time and when does this change occur? ● How far back is data relevant to today’s problem? ● What happens when our predictors change again in the future? ● What if this is all happening rapidly… will it scale?
  • 8. Enter Online Random Forest ● Input is a single new observation ● Trees learn incrementally on this new data ● Trees are dropped from the forest based on performance and replaced a new “ungrown” tree
  • 9. Visualization of a single tree Accuracy on test cases: 75% 5, 6 0, 70 Pure data stop splitting
  • 10. Visualization of a single tree Accuracy on test cases: 55% 0, 70 2, 25 20,3 50 new observations have come and we create another split off the parent node’s left branch
  • 11. Tree gets pruned Accuracy on test cases: 55% … compare to Random variable and incorporate the age of the tree. Accuracy is TOO BAD. Prune the tree 0, 70 2, 25 20,3
  • 12. New Tree It’s a stump that hasn’t yet split any data. If asked for a classification request it will vote the prior probability calculated from the last 100 observations that the old pruned tree saw
  • 13. Online Random Forest ● By dropping trees that predict poorly we can adapt to change in important predictors ● If previous data is relevant to today’s problem, tree’s learned from it in the past. If it no longer becomes relevant it will be reflected in the accuracy and the tree will get prune
  • 14. Online Random Forest ● This process of incremental learning and dropping is constantly occurring so we can constantly adapt to a changing signal ● We built our Online Random Forest with scala’s actor framework ● We distribute our tree’s computations (and physical location) therefore we can handle high input data streams
  • 15. Example Stream
  • 16. Changing Feature Importance