Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sampling methods for counting temporal motifs

321 views

Published on

Slides from talk at ATD workshop.
October 11, 2018.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Sampling methods for counting temporal motifs

  1. 1. 1 Joint work with Paul Liu & Moses Charikar (Stanford) Sampling methods for counting temporal motifs Austin R. Benson · Cornell ATD Workshop · October 11, 2018 Slides. bit.ly/arb-ATD18 Paper. arXiv:1810.00980
  2. 2. Temporal network data is extremely common. 2 Private communication e-mail, phone calls, text messages, instant messages Public communication Q&A forums, Facebook walls, Wikipedia edits Payment systems credit card transactions, cryptocurrencies, Venmo Technical infrastructure packets over the Internet, messages over supercomputer
  3. 3. Existing methods for temporal analysis are insufficient. 3 1. Models for network growth Growth of academic collaborations,Internet infrastructure,etc.[Leskovec+ 07] 2. Sequence of snapshot aggregates Daily phone call graph [Araujo+ 14],weekly email snapshots [Xu-Hero 14] Modern temporal network datasets • fine-scale time resolution • high-frequency • many repeated edges
  4. 4. Motifs,or small subgraph patterns,are commonly used to analyze static (non-temporal) networks. 4 A B C 1. Common feature for anomaly detection, role discovery, and other network machine learning problems. [Noble-Cook 03; Sun+ 07; Henderson+ 12; Rohe-Qin 13; Rossi-Ahmed 15; Benson-Gleich-Leskovec 16] 2. Finding fundamental components of complex systems. [Milo+ 02] • Triangles in social networks. [Rapoport 53; Granovetter 73; Watts-Strogatz 98] • Bi-directed length-2 paths in brain networks. [Sporns-Kötter 04; Sporns+ 07; Honey+ 07]
  5. 5. Motifs are defined for temporal networks,but we do not have scalable algorithms for real-time analysis. 5 Temporal network motif. 1. Directed multigraph with k edges 2. Edge ordering 3. Max. time span δ = 25. Motif instance. k temporal edges that match the pattern that all occur within δ time. Definition from [Paranjape-Benson-Leskovec 17] Problem. We do not have scalable algorithms for counting these patterns, especially for real-time data analysis.
  6. 6. 6 How do we enable real-time motif analysis for high-throughput temporal network data?
  7. 7. Continuous input of event data Update graph representing relations between nodes based on events Update the embedding of each node based on changes to the graph structure Perform real-time statistical analysis of the node embedding as a multivariate time series Output statistically anomalous nodes or changes in the graph If needed, update types of graph structures used to embed nodes ATD: Statistically Principled Real-Time Detection of Anomalies for Temporal Network Data. 7 With Anil Damle (Cornell) and Yuekai Sun (Michigan)
  8. 8. Parallel sampling yields about two orders of magnitude speedup and enables otherwise infeasible computations. 8 Using backtracking algorithm from [Mackey+ 18] as a sub-routine. Time scale δ = 1 day. 16 threads. running time (seconds) dataset # temporal edges exact sampling parallel sampling error StackOverflow 47.9M 221.7 93.10 5.208 4.9% EquinixChicago 345M 481.2 45.50 5.666 1.3% RedditComments 636M X 6739 2262 – <latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit>
  9. 9. 9 Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25, s = 5). Sampling window length w > δ (w = 50). 25 0 25 50 75 100 1. How do we re-scale exact counts? 2. Motifs can cross sampling intervals. How do we mitigate this? 3. How do we choose sampling probabilities q? sample with prob.q1 sample with prob.q2 We find motifs in sampled windows and re-scale counts.
  10. 10. 10 Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25, s = 5). Sampling window length w > δ (w = 50). 25 0 25 50 75 100 We find motifs in sampled windows and re-scale counts. Theorem.If we sample window j with prob. qj, then upscaling each found motif instance by (1 – d(M) / w) / qj is an unbiased estimator, where d(M) is the duration of the motif instance M. duration d(M) = 32 – 16 = 16 motif instance M
  11. 11. 11 Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25). Sampling window length w > δ (w = 50). 25 0 25 50 75 100 We find motifs in sampled windows and re-scale counts. • Using multiple random shifts and averaging the estimates reduces variance by capturing motifs that cross sampling intervals. • s = 5, s = 8, s = 15 • Computation over each shift is parallelizable.
  12. 12. 12 Choose random negative shift s uniformly from [0, 1, …, δ - 1] (δ = 25, s = 5). Sampling window length w > δ (w = 50). 25 0 25 50 75 100 • Set qj for each shift. Larger qj ⟶ more computation but less variance. • Importance sampling. Only want to sample where motifs occur. • Heuristic. Make qj larger if more edges in sampling window. sample with prob.q1 sample with prob.q2 We find motifs in sampled windows and re-scale counts.
  13. 13. 13 Input. Temporal motif and maximum time scale δ . Output. Estimate of number of instances of the motif. 1. Sample shift s uniformly at random from [0, 1, …, δ - 1]. 2. Sample window jth window [(j – 1)w – s, jw –s – 1] with probability qj. 3. Upscale counts of motif instances in jth window by (1 – d(M) / w) / qj. 4. Repeat 1–3 for multiple shifts and output the mean of estimates. Key advantages. 1. Works in streaming setting. Faster & less memory intensive. 2. Can use (almost) any “exact counting” method for step 3. [Paranjape-Benson-Leskovec 17; Mackey+ 18; Liu-Benson-Charikar 18] 3. Can parallelize over shifts and sampling windows ⟶ exposes parallelism to otherwise sequential algorithms.
  14. 14. Parallel sampling yields about two orders of magnitude speedup and enables otherwise infeasible computations. 14 Using backtracking algorithm from [Mackey+ 18] as a sub-routine. Time scale δ = 1 day. 16 threads. running time (seconds) dataset # temporal edges exact sampling parallel sampling error StackOverflow 47.9M 221.7 93.10 5.208 4.9% EquinixChicago 345M 481.2 45.50 5.666 1.3% RedditComments 636M X 6739 2262 – <latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit><latexit sha1_base64="aWJw05Gu7W4ToL0DYQ/70JVM7t4=">AAAI33icfVVbb9s2FHazm6fd2vVte2EXOGgLW7Hs2okHFDDWrliHBu3m9AKEQUdJRxJhUlRIqnEq6Hlvw173O/Zr9m92KDupnWajAZM6POc7V54TFoIb2+//c23rgw8/+viT9qfeZ59/8eVX1298/cKoUkfwPFJC6VchMyB4Ds8ttwJeFRqYDAW8DOcP3P3LN6ANV/mhPSvgWLI05wmPmEXS6xvX/qYhpDyvLAtLwXRdaTKt6MlJyeKaROe/2qNWFboU4Hk7ZIdQWQrLUX0p82pYVxHKlXnO85RYLoHcNhCpPDZ3akIpoZHksRO+LfSdatgb1V7MLFptHdQ2sSALpZkgEKdgkAYLFrk7wyQGAUF3SMGQQYBYp4HWSjcKVvgemVkWzZ+ix4lQpwR57u35kwPcB4PA3yNu7ZDJ0A/6uI/8QX/f8fgT2iHLRalHfjwpec4XDzIMU6qQYXhvdNCA7Qf+YAVyb+SPliDj8Rj3wB9ugPwKccztAyUl5NY5NR6OG5CKhgl5VTvK3nBCGtvGA7f3eoRcIBAaKmuVbPyikMcXKfJeX9/u+/1mkfcPweqw3VqtZ69vfHyTxioqnSmRYMYcBf3CHldMYxIFYHZLAwVGjqVwhMecSTDHVVNeNekgJSYJhjpRuSUN1VsXQRzNzjZQzo1dbFJDpeZ4Y2rP29Rpk/3jiudFaSGPliqTUhCriKtaEnMNkRVnZFOv5fO33ZxHkGgWdZk0ktmsW3BnZ9fO3/ZSzYqsK9kcIhDiHWlplRMXPNRMnzkX1KnphoicalVi7XYLZi3o3KC81XzRNRkrwHQTbrsRE5H7jp1MIZSVTM/Nf6H6EizDyyZyAmx1WCYWsEDw1UB8a79/KxSod53DZpBqgLyums3xnGbcwiWeUJRQV+5/jcPrkMzawny/u2th4Rv3JGARZSxPwY+U3D0pwbjnb3aD8WgymOwakBzfTIhNQfZOuc16zokez3sh9hLQDd9wb3u5edQFlGGvcfHxaCpUyATFT+rEppCbUsM0VgILYIqdJlIx3KcaBFucyyo0frOIjg6D48olzhXARpafHc5Y7oKrIYdTdEAyfA00YZKLsxgShr2orqhJzs+bRWISVxW111lXZjCDEN/v+5MuNidUimUhsORRgV2YxEFsOonYNLcLBzVdClfm7hG+tdFxfdmph4CPTMPsTIZKPEKXqiWKqaunB0/qKncqJK8rWVcczaUzsFcxIyG+LBKuRFY6nMCsDDGdtnQpvVrBZQ2zRwcuJOcKDoON8FXhoq6MeKfEMS+lq8fI6WLARJGx+p2pvz2+FPU4FcCjrLeM/VU3mGiD7WWzP0gHs55lOeOpRE10WVUODhuorOiSXr9XFvIJTr/4KonVRb2p4i5dhEwfYfHRLFSLir5x/x2PZq7zkgx4mlnsrnujwpIOOcyA4HAqcVqhmEfn2CH6/mAEi85F8+6Qhzi5WR4BCcGe4vt1vASVEdOE0Vuq6njY6B1Ar+8HIDvn0rNMaYyOG3IqJ1hUREBiieExOIk1v7aD+gIEB8Dwf0F040mDUrso4BgJLg+N9w8vBjgs/eCXwfb0h9VAabe+bX3Xut0KWnutaeun1rPW81a09c3WdOvx1s9t1v69/Uf7zyXr1rWVzM3Wxmr/9S+1GQMs</latexit>
  15. 15. 15 THANKS! Slides. bit.ly/arb-ATD18 Paper. arXiv:1810.00980 Austin R. Benson http://cs.cornell.edu/~arb @austinbenson arb@cs.cornell.edu Sampling methods for counting temporal motifs Where we are headed. 1. Node-level estimates instead of graph-wide estimates. 2. Theory on how good the importance sampling works given some structure. 3. Models for how temporal networks evolve. Supported by NSF ATD Award 1830274.
  16. 16. Variance results. 16 To get average squared error (✏CM)2 , we need to set the parameters as: E[kˆYsk2 2] E[kYsk2 2] C2 M + 1 c 1  b✏2 , where b is the number of shifts. The first term in the left-hand side combines (i) a natural measure of sparsity of the distribution of motifs with (ii) the extent of correlation between the sampling probabilities qj and the (weighted) motif counts for intervals Ys,j.<latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit><latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit><latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit><latexit sha1_base64="8zL3UR/f4DVWXdM4/82OOFTqrxM=">AAAJNnicfVVLk9tEEHYMBCNeCTlymbA2tUlsr+3UkkdVqrZISJGqpAjYedXKa2akljTxaKSdGcV2pvQ7uMK/4K9w4UZx5SfQI3mz690FHezRqPvr19fdLBdcm8HgjwvN997/4OKHrY+8jz/59LPPL13+4rnOChXAsyATmXrJqAbBJTwz3Ah4mSugKRPwgs3vu+8v3oDSPJMTs8phmtJY8ogH1ODV7HLT82XGZQjSkElGYjCEojyNgejDgioICSiVKdLe9iHXXGSS3J89uXYwanfJAogElDAZ0ahoEvByqmgKBi0Squ96PoOYS0sFj+X10vMjRQPrp9QkjNnvyn1fPAdliJ9QY1+VM+0r9z4bHYympEfOCr7aECktunIwKskNUiMPSxv0hiX52hdwSNiRxwejrueDDN/5sUhAAWmzNuHauU1kkTJQJIuITnhkdN+b4G3ElcawQKWEy0pOQGR6CZUh0TwEL8hShonXZJtfI5RIagpFBUmB6gINODhMiOZm5c4OIMSSKs4Kl313l2aGR5osuEm8bY4oTgiWxtUDPweZUiCqWhEGZgFQ+6FpiuyQMclVxijjghuObrQPZ6/bxLnnarG9AB4nBsJrtRlEK6TRJMJycolhvaECdV7NrO6+Ltv92aWtQX9QPeTsYbg+bDXWz9PZ5YtX/DALihS9DQTVen84yM3UUmV4IADrXWjIaTBHOu3jUSI19NRWrC1JB2/Cypkow2irW++kCuIoutpAsYayQlC13LxlWTbHL7r0vE2bJro9tVzmBSY0qE1GhXCEdc2A1VAQGLEim3YNn7/tSh6AI1WXptrxsJtz52fXzN/2YkXzpJvSOQQgxPFV7ZVTF5wpqlYuhGyhuwyRY4XpD3U3pwZzLzXqIxWWXZ3QHHQ34qYbUBG499Dp5CIzKVVz/V+ofWw0ih+rzAkwdlJEBn6CsLTYt1dvD64ygXZPSiAtYoUkKm3152QWCTdwSoaJAkrrfk9IeB2SGJPruzs7BpZ9bRAblgF2Qwx97ISdwwK0Y6reGX6ze2d0Z0dDynGYMORv2nMU77kgelz2GI4oUJXczVtb9Z/nu4RSHGEuP54fC6S2wE42vlPbA+maai/MBBJgDwdYkIVwz3f9sTzSzdD5TRLtT4ZT6wrnCLBR5aeTMZUuuQokLDCAFBvH+hFNuViFENFCmNL6Ojo6b5JER44Vpdc5aUxjBSG8N+jf6Qap6zGkhUDKowGz1JGD2AwSsX1plg5qr1a2+vo+9trutDwd1APAJlMwXqUsEw8xJFuj6NL+8ORxaaUzkfLSpqXl6K4/BnOeMF6Ep1XYWmVtwymMC4blNNWoOt/AaQvjh09cSo4MTIYb6bNsWVotjo044VrbPirrrUBFntDy2NWfH53KehgL4EHSq3N/3hcstMbxsjkfUgdzssrpmMcpWvJrVjk467PU+vV9eYYW6WNcquF5GusP5aaJ6/6SUbWP5PMTli2t/8b9djw/UYUAklSjGafrrd3ckA5x64YGpsDtgWqeP8cJMeiPdmHZIUdPhzzA7UFlAO92AcpW415XafRqUx2PkAqgN+gPIe0caY+TTGF23NrIjtdZvclQ40RcW8PyHQgugJv/C6KqSCqU0mUB18jw9NI4e3g+6g/RvR9HW3vfrhdKq/Fl46vGdmPYuNXYa3zfeNp41giah81fmr82f2v93vqz9Vfr71q0eWGtc6Wx8bT++Rf5N0AW</latexit> • Let Ys,j be the number of motif instances in the jth window with shift s. • Let ˆYs,j = Ys,j/ p qj. • Then V[Z]  E[kˆYsk2 2] E[kYsk2 2]+ 1 w/ 1 C2 , where C is the true total number of motif instances.<latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit><latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit><latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit><latexit sha1_base64="bpzMp92UIkFl0xi4svc0PzVSXkg=">AAAIlnicfVVbb9s2FFayrfO0S9v1ZcBe2MUGhk52bBdZ2wEdgvWyFWixbE7SdpabUdKRxZqkFJKqnRL6kXvcL9nrDi17sdJ0AizR5PnOx3OPCs606ff/3tr+4MOPrnzc+sT/9LPPv7h67fqXxzovVQxHcc5z9SKiGjiTcGSY4fCiUEBFxOF5NHvgzp+/AaVZLg/NWQETQaeSpSymBrdOrm+9DiOYMmmZAcHeQuWHbkWegiHtlydWB6+rNomAmAyILEUEiuQpEblhKWFSGypj0LhaCrRft01G5kwm+Rw/uNYZS1GTbvc2FYcZNfZlVasn98mKiOySUJ8qY09PkHWNOMxAIkRQk0WRPa7Gf0xIyOGUrLceVeOQH4MyZK1Xh8r9Pxm+Gk5I9xLBl02R70iYKhrbQWXn7hIJcEMROKjIg1fDduDPM1Bo3oM2YXppqVElvnJD+fu9ggaATM49e3Jtp9/rLx/y7mKwWux4q+fg5PqVG2GSx6UAaWJOtR4P+oWZWKoMi7mLVamhoPGMTmGMS0kF6IldZkVFOriTkDRX+JOGLHf9TQjqUfSsocUaGpWcqkVzN8rzGZ7oyvebnCa9O7FMFqUBGdeUacnRMcQlG0mYgtjwM9LkNWz2NpAsBuf0gArt4hMUzN0zMLO33amiRRYIOoMYOD/fqm/l4JxFiqozZ0I+10GEmqcqL2Wig4IaA0pqxBvFFoHOaAE6SJkJYspj9z9xmILnRlA10+/T2hNgKB4uPcfB2MMyNfA7JJVVkNy8278ZceTdlMDUmCoAWdnlx8nMM0yACzIRL6Gy7r0h4XdIZkyhf9jdNbDoYRLFM1jEGZVT6MW52D0tQbuq1buD7/fuDe/takwsLO4Ia1l0Xbl1nRFdJrsRtgBQS7nbd3bqjx86h1JsEc4/fjjleUQ5VpIJHWwfpC4V7Cc5xwTYxwYR5wncDxVwulhjc7x8M4nGh4OJdYFzCdCI8sHhiErnXAUS5miAoFgNYUoF42cJpLTkprKhTtfrZpLo1GVF5Xc2yTRGEJL7/d69IBYMSTEtOKY8EpiFTp2KppGoO5Rm4VTt12Crb42x1vYm1UWjHgIWmYLRmYhy/hhNsrUWXdlfnz2trHQUglVWVFjUCB+BuUwYN5KLkGgFWXE4wKiMMJymdCG9nOAiw+jxM+eSNcHhoOE+Gy0qq/k5iROu0fYJSjofUF5ktDq/6p9PLng9mXJgcdatfX/ZCQZaY3tp9gfh1GxGWYzYVCBTWGeVU2fDSNiw3q/eSQvxFIdWchlidVA1KW6Fi4iqMSZfmEX5woZv3Lvjh5kqOZAM2DQz2F3v7BWGdNwQITQ2JfZrhPnhDDtEvzfcg0WHrJ8Oecjq1o0Dz8yxfp0sQTKil270a6qOT8hSQbffG4DorNGjLFfoHSanJK/nIQccfpol4BAbdu0Mqv+U4AC4/b9K1NKSpZbKeQHHyODi0Hh3cTzsDfB6vw139n9aDZSW97X3jfetN/DuePveL96Bd+TFW39t/bPtbW+1vmr92HrU+rkW3d5aYW54jad18C9YvAFC</latexit>
  17. 17. Correlation results. 17 Actual motif counts are correlated with sampling probabilities that are chosen proportional to the fraction of temporal edges in a sampling window.

×