SlideShare a Scribd company logo
1 of 40
Time-evolving Relational Classification
and Ensemble Methods
Ryan A. Rossi
Jennifer Neville
 Given a graph G and a set of attributes X for each
           node, the task is to infer Y the class labels of the nodes
                                 class
                            n1
                            n2
  Training                  n3
                             .
                             .
.3 … B            .1 … B     .

                            Ignores temporal information!                       .1 … B    .4 … A

         .5 … A
                           Learning      Model     Inference
                                                                                     .3 … A        .9 … A
.2 … A            .4 … A

                                                 .1 … B       .4 … A
                                                   ?            ?

                                                          ?              ?
                                                       .3 … A          .9 … A


                                                       Testing
Problem
Attribute prediction in time-varying networks

                     Previous work
Contributions        - only models temporal edges
                     - predicts only static attribute
Consider the space   - only single-classifier
                     of temporal-relational representations
                     - limited representation!
1. Propose temporal-relational classification framework
2. Temporal-relational ensembles
What types of information can change?
Edges
 Most real networks are dynamic
    • social, information, biological,...




⋯                                           ⋯        ⋯

                                                    Time
Edges
 Most real networks are dynamic                                    Node Attributes



             .3 … C                           .2 … C                               .3 … C
                                   .1 … C                               .6 … L

                 .5 … L                           .2 … C                               .2 … C

                          .5 … L                           .3 … L                               .6 … L
    .6 … L                           .5 … L                               .2 … C

             .7 … L                           .4 … L                               .5 … L




⋯                                                                   ⋯                                    ⋯

                                                                                                    Time
Edges
 Most real networks are dynamic       Node Attributes
                                       Edge Attributes



       - … 5           + … 2                     + … 8
               + … 8
       + … 4
                               + … 2                     + … 1




⋯                                      ⋯                          ⋯

                                                                 Time
 Attributes of linked nodes are correlated


 Temporal locality: Recent events are more influential than
  distant ones                                     Time

         u       u   u    u     u     u       u      u
         τ2          τ2   τ1    τ1    τ1      τ1     τ1

         v       v   v    v     v     v       v      v

 Temporal recurrence: A regular series of events is more likely
  to indicate a stronger relationship than an isolated event
                                                   Time

         u       u   u    u     u     u       u       u

         v       v   v    v     v     v       v       v
How to represent the data with these in mind?
X1 X2 X3 X4 ... Xm
                                   Input                                           ...
      Temporal-Relational                                                          ...
                                                                                   ...
    Classification Framework                                                       …
                                                                                   …
                                             Dt=                                   ...
                                                                                   …
                                                                                   ...
Edges     Attributes      Nodes                                        ⋮ ⋮ ⋮ ⋮ ⋱⋮
                                                            Gt                Xt
                                                                     where t = 1,...,tmax
                                  - Single timestep
    Temporal Granularity          - Window of timesteps
                                  - Union (all timesteps)

                                                        For each: edges, attrs,...
                                                        1. Select the past information
TIME
     Window   Window   Window




G1    G2                 G3
X1 X2 X3 X4 ... Xm
                                   Input                                           ...
      Temporal-Relational                                                          ...
                                                                                   ...
    Classification Framework                                                       …
                                                                                   …
                                              Dt=                                  ...
                                                                                   …
                                                                                   ...
Edges       Attributes    Nodes                                        ⋮ ⋮ ⋮ ⋮ ⋱⋮
                                                            Gt                Xt
                                                                     where t = 1,...,tmax
                                  - Single timestep
    Temporal Granularity          - Window of timesteps
                                  - Union (all timesteps)

                                                        For each: edges, attrs,...
                                  - Exponential         1. Select the past information
        Temporal Influence
                                      ⋮                 2. Select weighting
                                  - Uniform
TIME
                              Window




   G1                 G2                  G3              Summary Graph


                    Temporal Granularity               Temporal Influence

1. Create summary graph
2. Assigns weights to edges (recent/frequent events larger weights)
• Weight functions: Exponential, linear, uniform,...
X1 X2 X3 X4 ... Xm
                                   Input                                           ...
      Temporal-Relational                                                          ...
                                                                                   ...
    Classification Framework                                                       …
                                                                                   …
                                              Dt=                                  ...
                                                                                   …
                                                                                   ...
Edges       Attributes    Nodes                                        ⋮ ⋮ ⋮ ⋮ ⋱⋮
                                                            Gt                Xt
                                                                      where t = 1,...,tmax
                                  - Single timestep
    Temporal Granularity          - Window of timesteps
                                  - Union (all timesteps)

                                                        For each: edges, attrs,...
                                  - Exponential         1. Select the past information
        Temporal Influence
                                      ⋮                 2. Select weighting
                                  - Uniform
                                                        3. Select relational classifier


                                  - Relational Probability Trees
     Relational Classifier
                                      ⋮
                                  - Relational Bayes Classifier


         Predictions
The attribute values are weighted by the product of their attribute weight and the
corresponding link weight
  TVRC (weights only edges)                     TENC (edges & attributes)




Weighted mode feature:
    TVRC  red (τ3 )                                  TENC  blue(τ2)
 Learn model on Dt (or set of timesteps)  apply it to Dt+1
 Use k-fold cross-validation to learn “best” model (k=10)
 Avg AUC over 10 trials
 PyComm (Python Developer Communication Network)
  • Extracted emails and bug discussions from the open-source python
    development environment
 Cora Citation Network

  PyComm (Communication Network)                 Cora Citiation Network
  Developers: 1,914                        Papers: 16,153
  Emails: 13,181                           References: 29,603
  Bug Messages: 69,435                     Authors 21,976
  Timespan: Feb 2007 – May 2008            Timespan: 1981 - 1998

               In this talk we focus on the more difficult dynamic prediction task


    Dataset    Prediction Task
  PyComm       Is developer productive or not in t+1? (closes bug) [Dynamic]
  Cora         1. Is paper AI or not?
               2. Predict paper topic (1 out of 7 ML topics) [Static]
The simplest temporal
                                      relational model still
                                      outperforms the others
      1.00




                                       TVRC         Most primitive
                                       RPT
                                       Intrinsic    temporal-relational model
      0.98




                                       Int+time
                                       Int+graph
                                       Int+topics
AUC
      0.96




                                                     Important to moderate the
                                                     relational information with
      0.94




                                                     the temporal info!
      0.92




             TVRC RPT   DT w/ additional attrs
Complexity in representation/accuracy     Temporal influence of edges
                                                             and attributes
      0.95




                                              TENC
                                              TVRC           Temporal influence of edges
                                              TVRC+Union
      0.90




                                              Window Model   strictly based on
                                              Union Model    temporal-granularity
                                                                Very efficient
      0.85
AUC

      0.80




                                Main finding: Accuracy generally
      0.75




                                increases as more temporal information is
                                included in the representation
      0.70
      0.65




             T=1          T=2        T=3          T=4
Can we increase accuracy by creating ensembles
    using temporal-relational information?
Key Idea: Introduce variance in both the temporal and
relational information


1.   Transform the temporal-relational representation using
     some process (e.g., randomization)
2. Learn models (on different training data)
3. Consider a weighted vote from model predictions
1. Transforming the Temporal Nodes and Links
   • sampling nodes and edges from discrete timesteps

2. Transforming the Temporal Feature Space
   • randomization of attributes (locally)
   • varying temporal influence or granularity
   • resample from temporal features
                Many possibilities!
3. Adding Temporal Noise or Randomness
   • randomly permute node feature values
   • randomly permute links across time

4. Transforming Time-varying Labels
   • randomly permuting previously learned labels at t-1 (or more distant) with
     the true labels at t

5. Multiple Classification Algorithms
   • sampling from a set of classifiers (RPT, RBC,...)
TIME
               .3 … C                           .2 … C                               .3 … C
    .2 … C                           .1 … C                               .6 … L

                   .5 … L                           .2 … C                               .2 … C

                            .5 … L                           .3 … L                               .6 … L
      .6 … L                           .5 … L                               .2 … C

               .7 … L                           .4 … L                               .5 … L


⋯                                                                     ⋯                                    ⋯

1. Select edge                                                   Randomization                      Edges
2. Randomly permute it over time

               .3 … C                           .2 … C                               .3 … C
    .2 … C                           .1 … C                               .6 … L

                   .5 … L                           .2 … C                               .2 … C

                            .5 … L                           .3 … L                               .6 … L
      .6 … L                           .5 … L                               .2 … C

               .7 … L                           .4 … L                               .5 … L


⋯                                                                     ⋯                                    ⋯
TIME
               .3 … C                           .2 … C                               .3 … C
    .2 … C                           .1 … C                               .6 … L

                   .5 … L                           .2 … C                               .2 … C

                            .5 … L                           .3 … L                               .6 … L
      .6 … L                           .5 … L                               .2 … C

               .7 … L                           .4 … L                               .5 … L


⋯                                                                     ⋯                                    ⋯

1. Select node and attribute                                     Randomization                      Edges
2. Randomize its values over time                                                                   Attributes

               .3 … C                           .2 … C                               .3 … C
    .2 … C                           .1 … C                               .6 … L

                   .5 … L                           .2 … C                               .2 … C

                            .5 … L                           .3 … L                               .6 … L
      .6 … L                           .5 … L                               .2 … C

               .5 … L                           .7 … L                               .4 … L


⋯                                                                     ⋯                                    ⋯
TVRC improvement significant at p < 0.05,
  16% reduction in error                              Temporal-relational ensemble
                                                      Relational ensemble
      1.00




                                                      Non-relational ensemble
                                     TVRC
                                     RPT
                                     DT
      0.98




                                              Main Findings:
                                              Temporal-relational ensembles can
AUC
      0.96




                                              improve over relational-ensembles,
                                              even using the minimum temporal
                                              information!
      0.94
      0.92




             T=1   T=2   T=3   T=4   Avg
1.0




                                                       TVRC
                                                       RPT
                          Slight improvement           DT
                       Significant improvement!
      0.9




                                                              Main Finding:
                                                              Temporal-relational ensembles
AUC

      0.8




                                                              perform significantly better
                                                              than the others when there
                                                              are more dynamics
      0.7
      0.6




            Communication   Team      Centrality   Topics




                                                              Relatively stationary attributes
               Frequently changing attributes
Main Contributions
 Proposed a general time-evolving relational classification
  framework for modeling temporal edges, attributes, and
  nodes
 Proposed classes of temporal ensemble methods

Main Findings
 Temporal-relational models are more accurate than relational
  and non-relational models
 Incorporating temporal information can increase accuracy of
  classification for both static and dynamic prediction tasks
 Temporal-relational ensembles yield better accuracy than
  relational and non-relational ensembles
 Systematically investigate the ensemble methods and
  identify when or where to apply each strategy
Thanks!


          Questions?
         rrossi@purdue.edu
http://www.cs.purdue.edu/homes/rrossi
U. Sharan et al. (ICDM 2008)


Differences
 Our methods generalize for any dynamics
  •   They model only edges
 Propose and use temporal-relational ensembles
 Static and dynamic prediction tasks
X1;t X2;t X3;t X4;t ... Xm;t
                                             …
                                             ...
                                             …

                                                        ⇒
                                             ...
Dt =                                         …
                                             ...
                                             ...
                                             ...
                          ⋮   ⋮    ⋮    ⋮    ⋱ ⋮            ⋮

          Gt                           Xt                   Yt+1

       E(Yt+1 | Gt ,Gt-1,..., Xt , Xt-1,...)
TIME
                  Window   Window   Window




   G1              G2                  G3           Summary Graph


        2. Temporal Granularity                 3. Temporal Influence

1. For each piece of relational data
  • attributes, edges, or nodes
2. Select the amount of past information to use
3. Select how the past information is weighted
  • Weight function and decay parameter
TIME
                                 Window




Xi1                     Xi2                     Xi3          Summary Attributes



        Select the timesteps for learning                    Temporal weighting
                                                             /summarization
      We can also do this for each attribute!
where t = 1,...,tmax

  Temporal-Relational Classification Framework

Relational Information       Edges     Attributes       Nodes


Temporal Granularity        Timestep       Window          Union


Temporal Influence               Uniform       Weighting



Relational Classification            RPT      ⋯            RBC



                 Predictions
Temporal-Relational Model




Relational Information       Edges     Attributes   Nodes




Temporal Granularity        Timestep       Window     Union




Temporal Influence               Uniform       Weighting




Relational Classification            RPT      ⋯       RBC
Previous example was actually TVRC




Relational Information     Edges     Attributes   Nodes




Temporal Granularity      Timestep       Window     Union




Temporal Influence             Uniform       Weighting




Relational Classifier              RPT      ⋯       RBC
Let Gt = (Vt , Et ,Wt E ) be the relational graph at time step t

We define the summary graph GS(t ) = (VS(t ), ES(t ),WS(t) ) as the weighted
                                                                E

sum of graphs up to time t as follows:


         VS(t ) = V1 ÈV2 ÈÈVt
          ES(t ) = E1 È E2 ÈÈ Et                           t
         WS(t ) = a1W1E + a 2W2E ++ atWt E = å K E (Gi ;t, q )
           E

                                                           i=1
 The α weights determine the contribution of each snapshot

 We use the kernel function K with parameters θ to determines the influence of
  each edge in the summary

 Weights can be viewed as probabilities that a relationship (or attribute value) is
  still active at the current time step t, given that it was observed at time (t-k)
Attribute Weighting and Summarization




Graph Weighting and Summarization
Exponential
                   t1      t2      t3     t4                     Weighting  (1- l )t-i l

Node Attribute     τ1      τ2     τ2      τ1        ⇒                 τ1: (1−λ)3λ + λ
                                                                      τ2: λ((1−λ)2 + (1−λ))



                                                                      ω
Edge Attribute     τ1      τ2             τ1
                                                    ⇒                 τ1: (1−λ)3λ + λ
                                                                      τ2: (1−λ)λ




Edge                                                ⇒              ω = θ(1 + (1−θ) + (1−θ)3)



Weights can be viewed as probabilities that a relationship (or attribute value) is still
active at the current time step t, given that it was observed at time (t-k)

More Related Content

Similar to Time-Evolving Relational Classification and Ensemble Methods

EarGram: an Application for Interactive Exploration of Large Databases of Aud...
EarGram: an Application for Interactive Exploration of Large Databases of Aud...EarGram: an Application for Interactive Exploration of Large Databases of Aud...
EarGram: an Application for Interactive Exploration of Large Databases of Aud...Gilberto Bernardes
 
Situation recognition acm mm 121029
Situation recognition acm mm 121029Situation recognition acm mm 121029
Situation recognition acm mm 121029Ramesh Jain
 
Lesson 13: Related Rates of Change
Lesson 13: Related Rates of ChangeLesson 13: Related Rates of Change
Lesson 13: Related Rates of ChangeMatthew Leingang
 
Lesson 13: Related Rates of Change
Lesson 13: Related Rates of ChangeLesson 13: Related Rates of Change
Lesson 13: Related Rates of ChangeMatthew Leingang
 
Automatic generation of domain models for call centers
Automatic generation of domain models for call centersAutomatic generation of domain models for call centers
Automatic generation of domain models for call centersDavid Przybilla
 

Similar to Time-Evolving Relational Classification and Ensemble Methods (8)

Notes
NotesNotes
Notes
 
EarGram: an Application for Interactive Exploration of Large Databases of Aud...
EarGram: an Application for Interactive Exploration of Large Databases of Aud...EarGram: an Application for Interactive Exploration of Large Databases of Aud...
EarGram: an Application for Interactive Exploration of Large Databases of Aud...
 
Im2013vit
Im2013vitIm2013vit
Im2013vit
 
Situation recognition acm mm 121029
Situation recognition acm mm 121029Situation recognition acm mm 121029
Situation recognition acm mm 121029
 
Lesson 13: Related Rates of Change
Lesson 13: Related Rates of ChangeLesson 13: Related Rates of Change
Lesson 13: Related Rates of Change
 
Lesson 13: Related Rates of Change
Lesson 13: Related Rates of ChangeLesson 13: Related Rates of Change
Lesson 13: Related Rates of Change
 
Automatic generation of domain models for call centers
Automatic generation of domain models for call centersAutomatic generation of domain models for call centers
Automatic generation of domain models for call centers
 
Weighted Lasso for Network inference
Weighted Lasso for Network inferenceWeighted Lasso for Network inference
Weighted Lasso for Network inference
 

Recently uploaded

Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...amber724300
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Nikki Chapple
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
Why Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionWhy Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionDEEPRAJ PATHAK
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 

Recently uploaded (20)

Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
 
A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
Why Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionWhy Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile Evolution
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 

Time-Evolving Relational Classification and Ensemble Methods

  • 1. Time-evolving Relational Classification and Ensemble Methods Ryan A. Rossi Jennifer Neville
  • 2.  Given a graph G and a set of attributes X for each node, the task is to infer Y the class labels of the nodes class n1 n2 Training n3 . . .3 … B .1 … B . Ignores temporal information! .1 … B .4 … A .5 … A Learning Model Inference .3 … A .9 … A .2 … A .4 … A .1 … B .4 … A ? ? ? ? .3 … A .9 … A Testing
  • 3. Problem Attribute prediction in time-varying networks Previous work Contributions - only models temporal edges - predicts only static attribute Consider the space - only single-classifier of temporal-relational representations - limited representation! 1. Propose temporal-relational classification framework 2. Temporal-relational ensembles
  • 4. What types of information can change?
  • 5. Edges  Most real networks are dynamic • social, information, biological,... ⋯ ⋯ ⋯ Time
  • 6. Edges  Most real networks are dynamic Node Attributes .3 … C .2 … C .3 … C .1 … C .6 … L .5 … L .2 … C .2 … C .5 … L .3 … L .6 … L .6 … L .5 … L .2 … C .7 … L .4 … L .5 … L ⋯ ⋯ ⋯ Time
  • 7. Edges  Most real networks are dynamic Node Attributes Edge Attributes - … 5 + … 2 + … 8 + … 8 + … 4 + … 2 + … 1 ⋯ ⋯ ⋯ Time
  • 8.  Attributes of linked nodes are correlated  Temporal locality: Recent events are more influential than distant ones Time u u u u u u u u τ2 τ2 τ1 τ1 τ1 τ1 τ1 v v v v v v v v  Temporal recurrence: A regular series of events is more likely to indicate a stronger relationship than an isolated event Time u u u u u u u u v v v v v v v v
  • 9. How to represent the data with these in mind?
  • 10. X1 X2 X3 X4 ... Xm Input ... Temporal-Relational ... ... Classification Framework … … Dt= ... … ... Edges Attributes Nodes ⋮ ⋮ ⋮ ⋮ ⋱⋮ Gt Xt where t = 1,...,tmax - Single timestep Temporal Granularity - Window of timesteps - Union (all timesteps) For each: edges, attrs,... 1. Select the past information
  • 11. TIME Window Window Window G1 G2 G3
  • 12. X1 X2 X3 X4 ... Xm Input ... Temporal-Relational ... ... Classification Framework … … Dt= ... … ... Edges Attributes Nodes ⋮ ⋮ ⋮ ⋮ ⋱⋮ Gt Xt where t = 1,...,tmax - Single timestep Temporal Granularity - Window of timesteps - Union (all timesteps) For each: edges, attrs,... - Exponential 1. Select the past information Temporal Influence ⋮ 2. Select weighting - Uniform
  • 13. TIME Window G1 G2 G3 Summary Graph Temporal Granularity Temporal Influence 1. Create summary graph 2. Assigns weights to edges (recent/frequent events larger weights) • Weight functions: Exponential, linear, uniform,...
  • 14. X1 X2 X3 X4 ... Xm Input ... Temporal-Relational ... ... Classification Framework … … Dt= ... … ... Edges Attributes Nodes ⋮ ⋮ ⋮ ⋮ ⋱⋮ Gt Xt where t = 1,...,tmax - Single timestep Temporal Granularity - Window of timesteps - Union (all timesteps) For each: edges, attrs,... - Exponential 1. Select the past information Temporal Influence ⋮ 2. Select weighting - Uniform 3. Select relational classifier - Relational Probability Trees Relational Classifier ⋮ - Relational Bayes Classifier Predictions
  • 15. The attribute values are weighted by the product of their attribute weight and the corresponding link weight TVRC (weights only edges) TENC (edges & attributes) Weighted mode feature: TVRC  red (τ3 ) TENC  blue(τ2)
  • 16.  Learn model on Dt (or set of timesteps)  apply it to Dt+1  Use k-fold cross-validation to learn “best” model (k=10)  Avg AUC over 10 trials
  • 17.  PyComm (Python Developer Communication Network) • Extracted emails and bug discussions from the open-source python development environment  Cora Citation Network PyComm (Communication Network) Cora Citiation Network Developers: 1,914 Papers: 16,153 Emails: 13,181 References: 29,603 Bug Messages: 69,435 Authors 21,976 Timespan: Feb 2007 – May 2008 Timespan: 1981 - 1998 In this talk we focus on the more difficult dynamic prediction task Dataset Prediction Task PyComm Is developer productive or not in t+1? (closes bug) [Dynamic] Cora 1. Is paper AI or not? 2. Predict paper topic (1 out of 7 ML topics) [Static]
  • 18. The simplest temporal relational model still outperforms the others 1.00 TVRC Most primitive RPT Intrinsic temporal-relational model 0.98 Int+time Int+graph Int+topics AUC 0.96 Important to moderate the relational information with 0.94 the temporal info! 0.92 TVRC RPT DT w/ additional attrs
  • 19. Complexity in representation/accuracy Temporal influence of edges and attributes 0.95 TENC TVRC Temporal influence of edges TVRC+Union 0.90 Window Model strictly based on Union Model temporal-granularity  Very efficient 0.85 AUC 0.80 Main finding: Accuracy generally 0.75 increases as more temporal information is included in the representation 0.70 0.65 T=1 T=2 T=3 T=4
  • 20. Can we increase accuracy by creating ensembles using temporal-relational information?
  • 21. Key Idea: Introduce variance in both the temporal and relational information 1. Transform the temporal-relational representation using some process (e.g., randomization) 2. Learn models (on different training data) 3. Consider a weighted vote from model predictions
  • 22. 1. Transforming the Temporal Nodes and Links • sampling nodes and edges from discrete timesteps 2. Transforming the Temporal Feature Space • randomization of attributes (locally) • varying temporal influence or granularity • resample from temporal features Many possibilities! 3. Adding Temporal Noise or Randomness • randomly permute node feature values • randomly permute links across time 4. Transforming Time-varying Labels • randomly permuting previously learned labels at t-1 (or more distant) with the true labels at t 5. Multiple Classification Algorithms • sampling from a set of classifiers (RPT, RBC,...)
  • 23. TIME .3 … C .2 … C .3 … C .2 … C .1 … C .6 … L .5 … L .2 … C .2 … C .5 … L .3 … L .6 … L .6 … L .5 … L .2 … C .7 … L .4 … L .5 … L ⋯ ⋯ ⋯ 1. Select edge Randomization Edges 2. Randomly permute it over time .3 … C .2 … C .3 … C .2 … C .1 … C .6 … L .5 … L .2 … C .2 … C .5 … L .3 … L .6 … L .6 … L .5 … L .2 … C .7 … L .4 … L .5 … L ⋯ ⋯ ⋯
  • 24. TIME .3 … C .2 … C .3 … C .2 … C .1 … C .6 … L .5 … L .2 … C .2 … C .5 … L .3 … L .6 … L .6 … L .5 … L .2 … C .7 … L .4 … L .5 … L ⋯ ⋯ ⋯ 1. Select node and attribute Randomization Edges 2. Randomize its values over time Attributes .3 … C .2 … C .3 … C .2 … C .1 … C .6 … L .5 … L .2 … C .2 … C .5 … L .3 … L .6 … L .6 … L .5 … L .2 … C .5 … L .7 … L .4 … L ⋯ ⋯ ⋯
  • 25. TVRC improvement significant at p < 0.05, 16% reduction in error Temporal-relational ensemble Relational ensemble 1.00 Non-relational ensemble TVRC RPT DT 0.98 Main Findings: Temporal-relational ensembles can AUC 0.96 improve over relational-ensembles, even using the minimum temporal information! 0.94 0.92 T=1 T=2 T=3 T=4 Avg
  • 26. 1.0 TVRC RPT Slight improvement DT Significant improvement! 0.9 Main Finding: Temporal-relational ensembles AUC 0.8 perform significantly better than the others when there are more dynamics 0.7 0.6 Communication Team Centrality Topics Relatively stationary attributes Frequently changing attributes
  • 27. Main Contributions  Proposed a general time-evolving relational classification framework for modeling temporal edges, attributes, and nodes  Proposed classes of temporal ensemble methods Main Findings  Temporal-relational models are more accurate than relational and non-relational models  Incorporating temporal information can increase accuracy of classification for both static and dynamic prediction tasks  Temporal-relational ensembles yield better accuracy than relational and non-relational ensembles
  • 28.  Systematically investigate the ensemble methods and identify when or where to apply each strategy
  • 29. Thanks! Questions? rrossi@purdue.edu http://www.cs.purdue.edu/homes/rrossi
  • 30.
  • 31. U. Sharan et al. (ICDM 2008) Differences  Our methods generalize for any dynamics • They model only edges  Propose and use temporal-relational ensembles  Static and dynamic prediction tasks
  • 32. X1;t X2;t X3;t X4;t ... Xm;t … ... … ⇒ ... Dt = … ... ... ... ⋮ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮ Gt Xt Yt+1 E(Yt+1 | Gt ,Gt-1,..., Xt , Xt-1,...)
  • 33. TIME Window Window Window G1 G2 G3 Summary Graph 2. Temporal Granularity 3. Temporal Influence 1. For each piece of relational data • attributes, edges, or nodes 2. Select the amount of past information to use 3. Select how the past information is weighted • Weight function and decay parameter
  • 34. TIME Window Xi1 Xi2 Xi3 Summary Attributes Select the timesteps for learning Temporal weighting /summarization We can also do this for each attribute!
  • 35. where t = 1,...,tmax Temporal-Relational Classification Framework Relational Information Edges Attributes Nodes Temporal Granularity Timestep Window Union Temporal Influence Uniform Weighting Relational Classification RPT ⋯ RBC Predictions
  • 36. Temporal-Relational Model Relational Information Edges Attributes Nodes Temporal Granularity Timestep Window Union Temporal Influence Uniform Weighting Relational Classification RPT ⋯ RBC
  • 37. Previous example was actually TVRC Relational Information Edges Attributes Nodes Temporal Granularity Timestep Window Union Temporal Influence Uniform Weighting Relational Classifier RPT ⋯ RBC
  • 38. Let Gt = (Vt , Et ,Wt E ) be the relational graph at time step t We define the summary graph GS(t ) = (VS(t ), ES(t ),WS(t) ) as the weighted E sum of graphs up to time t as follows: VS(t ) = V1 ÈV2 ÈÈVt ES(t ) = E1 È E2 ÈÈ Et t WS(t ) = a1W1E + a 2W2E ++ atWt E = å K E (Gi ;t, q ) E i=1  The α weights determine the contribution of each snapshot  We use the kernel function K with parameters θ to determines the influence of each edge in the summary  Weights can be viewed as probabilities that a relationship (or attribute value) is still active at the current time step t, given that it was observed at time (t-k)
  • 39. Attribute Weighting and Summarization Graph Weighting and Summarization
  • 40. Exponential t1 t2 t3 t4 Weighting (1- l )t-i l Node Attribute τ1 τ2 τ2 τ1 ⇒ τ1: (1−λ)3λ + λ τ2: λ((1−λ)2 + (1−λ)) ω Edge Attribute τ1 τ2 τ1 ⇒ τ1: (1−λ)3λ + λ τ2: (1−λ)λ Edge ⇒ ω = θ(1 + (1−θ) + (1−θ)3) Weights can be viewed as probabilities that a relationship (or attribute value) is still active at the current time step t, given that it was observed at time (t-k)

Editor's Notes

  1. Now for an example....