Real-time Ranking with Concept Drift Using Expert Advice Hila Becker and Marta Arias Center for Computational Learning Systems Columbia University

Dynamic Ranking Continuous arrival of data over time Set of items to rank Dynamic features Adapt to Changes Given a list of electrical grid components, produce a ranking according to failure susceptibility

Continuous arrival of data over time

Set of items to rank

Dynamic features

Adapt to Changes

Given a list of electrical grid components, produce a ranking according to failure susceptibility

Problem Setting + + + - + - - - time + + + - + - - - + + + - + - - - + + + - + - - - ? . . . t-1 t 1 2 3 ? ? ? ? ? ? ? M + Feature Vector x = x 1 ,x 2 ,…,x n Label y

Challenges Changes in underlying distribution Hidden Concept drift Adapt learning model to improve predictions Finite storage space Sample from the data Discard old or irrelevant information

Changes in underlying distribution

Hidden

Concept drift

Adapt learning model to improve predictions

Finite storage space

Sample from the data

Discard old or irrelevant information

Concept Drift + + + + + - + + - - - - - - time

Ensemble Methods time

Weighted Expert Ensembles Associate a weight with each expert Measure of belief in expert performance Weights used in final prediction Use only the best expert Weighted average of predictions Update the weights after every prediction

Associate a weight with each expert

Measure of belief in expert performance

Weights used in final prediction

Use only the best expert

Weighted average of predictions

Update the weights after every prediction

Weighted Majority Algorithm e 1 . . . e 2 e 3 e N N Experts 1 0 0 1 ? w 1 *1 + w 2 *0 + w 3 *0 + . . . + w N *1 >0.5 <0.5 1 0 1

Modified Weighted Majority Different Constrains for data streams Incorporate new data Static vs. Dynamic set of experts Ranking Algorithm Loss function – 1-normalized average rank of positive examples Combine Predictions – weighted average rank

Different Constrains for data streams

Incorporate new data

Static vs. Dynamic set of experts

Ranking Algorithm

Loss function – 1-normalized average rank of positive examples

Combine Predictions – weighted average rank

Online Ranking Algorithm e 1 . . . e 2 e 3 e B w 1 w 2 w 3 w B ? F1 F4 F3 F2 F5 F4 F2 F1 F3 F5 F1 F3 F5 F4 F2 F1 F3 F4 F2 F5 F1 F3 F4 F2 F5 F3 F1 F4 F2 F5 e B+1 e B+2 w B+1 w B+2

Performance – Summer 05

Performance – Winter 06

Contributions Additive weighted ensemble based on the Weighted Majority algorithm Algorithm adapted to ranking Experiments on a Real-world datastream Outperform traditional approaches Explore performance/complexity tradeoffs

Additive weighted ensemble based on the Weighted Majority algorithm

Algorithm adapted to ranking

Experiments on a Real-world datastream

Outperform traditional approaches

Explore performance/complexity tradeoffs

Future Work Ensemble diversity control Exploit re-occurring contexts Use knowledge of cyclic patterns Revive old experts Change detection Statistical estimation of predicting ensemble size

Ensemble diversity control

Exploit re-occurring contexts

Use knowledge of cyclic patterns

Revive old experts

Change detection

Statistical estimation of predicting ensemble size

Ensemble Methods Static ensemble with online learners [Hulten ’01] Use batch-learners as experts Can use many learning algorithms Loses interpretability Additive ensembles Train an expert at constant intervals [Street and Kim ’01] Train an expert when performance declines [Kolter ’05]

Static ensemble with online learners [Hulten ’01]

Use batch-learners as experts

Can use many learning algorithms

Loses interpretability

Additive ensembles

Train an expert at constant intervals [Street and Kim ’01]

Train an expert when performance declines [Kolter ’05]

Ensemble Pruning Additive ensembles can grow infinitely large Criteria for removing experts Age - retire oldest model [Chu and Zaniolo ‘04] Performance Worst in the ensemble Below a minimal threshold [Stanley ’01] Instance-based Pruning [Wang et al. ’03]

Additive ensembles can grow infinitely large

Criteria for removing experts

Age - retire oldest model [Chu and Zaniolo ‘04]

Performance

Worst in the ensemble

Below a minimal threshold [Stanley ’01]

Instance-based Pruning [Wang et al. ’03]

Dealing with a moving set of experts Introduce new parameters B: “budget” (max number of models) set to 100 p: new models weight percentile in [0,100]  : age penalty in (0,1] If too many models (more than B), drop models with poor q-score, where q i = w i • pow(  , age i ) I.e.,  is rate of exponential decay

Introduce new parameters

B: “budget” (max number of models) set to 100

p: new models weight percentile in [0,100]

 : age penalty in (0,1]

If too many models (more than B), drop models with poor q-score, where

q i = w i • pow(  , age i )

I.e.,  is rate of exponential decay

Performance Metric ranking outages pAUC=17/24=0.7 0 8 0 7 0 6 1 5 0 4 1 3 1 2 0 1 8 7 6 5 4 3 2 1 1 2 3

Budget Variation

Data Streams Continuous arrival of data over time Real-world applications Consumer shopping patterns Weather prediction Electricity load forecasting Increased attention Companies collect data Traditional approaches do not apply

Continuous arrival of data over time

Real-world applications

Consumer shopping patterns

Weather prediction

Electricity load forecasting

Increased attention

Companies collect data

Traditional approaches do not apply