Seminar web data extraction                                                         1 / 26       Seminar web data extracti...
Seminar web data extraction > Frequent patterns & association rules                   2 / 26  Introduction    Focus of thi...
Seminar web data extraction > Frequent patterns & association rules                     3 / 26  Introduction & running exa...
Seminar web data extraction > Frequent patterns & association rules                 4 / 26  Measuring ‘interestingness’: s...
Seminar web data extraction > Frequent patterns & association rules                       5 / 26  Finding association rule...
Seminar web data extraction > Frequent patterns & association rules                       5 / 26  Finding association rule...
Seminar web data extraction > Frequent patterns & association rules                     6 / 26  Finding association rules:...
Seminar web data extraction > Frequent patterns & association rules                     6 / 26  Finding association rules:...
Seminar web data extraction > Introduction to uncertain data                     7 / 26  Introduction to uncertain data   ...
Seminar web data extraction > Introduction to uncertain data                    8 / 26  Existential probabilities    Exist...
Seminar web data extraction > Introduction to uncertain data                                     8 / 26  Existential proba...
Seminar web data extraction > Introduction to uncertain data                          9 / 26  Possible worlds        D = {...
Seminar web data extraction > Introduction to uncertain data                                             9 / 26  Possible ...
Seminar web data extraction > Mining uncertain data > Introduction                10 / 26 Introduction    Approaches to mi...
Seminar web data extraction > Mining uncertain data > Introduction                                11 / 26 Expected support...
Seminar web data extraction > Mining uncertain data > Introduction                                   11 / 26 Expected supp...
Seminar web data extraction > Mining uncertain data > Introduction                                     12 / 26 Expected su...
Seminar web data extraction > Mining uncertain data > Introduction                                         12 / 26 Expecte...
Seminar web data extraction > Mining uncertain data > Introduction                               13 / 26 Frequent itemsets...
Seminar web data extraction > Mining uncertain data > UF-growth                 14 / 26 Introduction to UF-growth   Aprior...
Seminar web data extraction > Mining uncertain data > UF-growth                 14 / 26 Introduction to UF-growth   Aprior...
Seminar web data extraction > Mining uncertain data > UF-growth                 14 / 26 Introduction to UF-growth   Aprior...
Seminar web data extraction > Mining uncertain data > UF-growth                                         15 / 26 UF-tree ex...
Seminar web data extraction > Mining uncertain data > UF-growth                             16 / 26 UF-tree example (2)   ...
Seminar web data extraction > Mining uncertain data > UF-growth                 17 / 26 UF-growth continued   Mining large...
Seminar web data extraction > Mining uncertain data > p-Apriori                  18 / 26 Introduction to p-Apriori        ...
Seminar web data extraction > Mining uncertain data > p-Apriori                  18 / 26 Introduction to p-Apriori        ...
Seminar web data extraction > Mining uncertain data > p-Apriori                         19 / 26 p-Apriori: advanced freque...
Seminar web data extraction > Mining uncertain data > p-Apriori                         19 / 26 p-Apriori: advanced freque...
Seminar web data extraction > Mining uncertain data > p-Apriori                              20 / 26 p-Apriori: pruning us...
Seminar web data extraction > Mining uncertain data > p-Apriori                                20 / 26 p-Apriori: pruning ...
Seminar web data extraction > Mining uncertain data > p-Apriori                                                   21 / 26 ...
Seminar web data extraction > Mining uncertain data > p-Apriori                    22 / 26 p-Apriori: finding frequent patt...
Seminar web data extraction > Summary & conclusion                                23 / 26 Summary & conclusion           D...
Seminar web data extraction > References                                       24 / 26 References I          C. C. Aggarwa...
Seminar web data extraction > References                                       25 / 26 References II          C. K. Chui, ...
Seminar web data extraction > References                                      26 / 26 References III          C. K. S. Leu...
Upcoming SlideShare
Loading in …5
×

Mining Uncertain Data (Sebastiaan van Schaaik)

1,718 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,718
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
70
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Mining Uncertain Data (Sebastiaan van Schaaik)

  1. 1. Seminar web data extraction 1 / 26 Seminar web data extraction: Mining uncertain data Sebastiaan van Schaik Sebastiaan.van.Schaik@comlab.ox.ac.uk 20 January 2011 Sebastiaan van Schaik
  2. 2. Seminar web data extraction > Frequent patterns & association rules 2 / 26 Introduction Focus of this presentation: mining of frequent patterns and association rules from (uncertain) data. Example applications: discover regularities in customer transactions; analysing log files: determine how visitors use a website; Based on: Mining Uncertain Data with Probabilistic Guarantees[9] (KDD 2010); Frequent Pattern Mining with Uncertain Data[1] (KDD 2009); A Tree-Based Approach for Frequent Pattern Mining from Uncertain Data[6] (PAKDD 2008). Sebastiaan van Schaik
  3. 3. Seminar web data extraction > Frequent patterns & association rules 3 / 26 Introduction & running example Frequent pattern (itemset): items that occurs sufficiently often. Example: {fever, headache} Association rule: a set of items values implying another set of items. Example: {fever, headache} ⇒ {nausea} Patient Diagnosis t1 Cheng {severe cold} t2 Andrey {yellow fever, haemochromatosis} t3 Omer {schistosomiasis, syringomyelia} t4 Tim {Wilson’s disease} t5 Dan {Hughes-Stovin syndrome} Yellow fever? t6 Bas {Henoch-Schnlein purpura} Running example: patient diagnosis database Sebastiaan van Schaik
  4. 4. Seminar web data extraction > Frequent patterns & association rules 4 / 26 Measuring ‘interestingness’: support & confidence Support of an itemset X : sup(X ): number of entries (rows, transactions) that contain X Confidence of an association rule X ⇒ Y : sup(X ∪ Y ) conf(X ⇒ Y ) = sup(X ) Sebastiaan van Schaik
  5. 5. Seminar web data extraction > Frequent patterns & association rules 5 / 26 Finding association rules: Apriori (1) Agrawal et al. introduced Apriori in 1994[2] to mine association rules: 1 Find all frequent itemsets X in database D (X is frequent iff i sup(Xi ) > minsup): 1 Candidate generation: generate all possible itemsets of length k (starting k = 1) based on frequent itemsets of length k − 1; 2 Test candidates, discard infrequent itemsets; 3 Repeat with k = k + 1. Sebastiaan van Schaik
  6. 6. Seminar web data extraction > Frequent patterns & association rules 5 / 26 Finding association rules: Apriori (1) Agrawal et al. introduced Apriori in 1994[2] to mine association rules: 1 Find all frequent itemsets X in database D (X is frequent iff i sup(Xi ) > minsup): 1 Candidate generation: generate all possible itemsets of length k (starting k = 1) based on frequent itemsets of length k − 1; 2 Test candidates, discard infrequent itemsets; 3 Repeat with k = k + 1. Important observation: all subsets X of a frequent itemset X are frequent (Apriori property). Used to purge before step (2). Example: if X = {fever} is not frequent in database D, then X = {fever, headache} can not be frequent. Sebastiaan van Schaik
  7. 7. Seminar web data extraction > Frequent patterns & association rules 6 / 26 Finding association rules: Apriori (2) Apriori continued: 2 Extract association rules from frequent itemsets X . For each Xi ∈ X : 1 Generate all non-empty subsets S of Xi . For each S: 2 Test confidence of rule S ⇒ (Xi − S) Sebastiaan van Schaik
  8. 8. Seminar web data extraction > Frequent patterns & association rules 6 / 26 Finding association rules: Apriori (2) Apriori continued: 2 Extract association rules from frequent itemsets X . For each Xi ∈ X : 1 Generate all non-empty subsets S of Xi . For each S: 2 Test confidence of rule S ⇒ (Xi − S) Example: itemset X = {fever, headache, nausea} is frequent, test: {fever, headache} ⇒ {nausea} {fever, nausea} ⇒ {headache} {nausea, headache} ⇒ {fever} {fever} ⇒ {headache, nausea} (. . . ) Sebastiaan van Schaik
  9. 9. Seminar web data extraction > Introduction to uncertain data 7 / 26 Introduction to uncertain data Data might be uncertain, for example: Location detection using multiple RFID sensors (triangulation); Sensor readings (temperature, humidity) are noisy; Face recognition; Patient diagnosis. Challenge: how do we model uncertainty and take it into account when mining frequent itemsets and association rules? Sebastiaan van Schaik
  10. 10. Seminar web data extraction > Introduction to uncertain data 8 / 26 Existential probabilities Existential probability: a probability is associated with each item in a tuple, expressing the odds that the item belongs to that tuple. Important assumption: tuple and item independence! Sebastiaan van Schaik
  11. 11. Seminar web data extraction > Introduction to uncertain data 8 / 26 Existential probabilities Existential probability: a probability is associated with each item in a tuple, expressing the odds that the item belongs to that tuple. Important assumption: tuple and item independence! Patient Diagnosis (including existential probabilities) t1 Cheng { 0.9 : a 0.72 : d 0.718 : e 0.8 : f } t2 Andrey { 0.9 : a 0.81 : c 0.718 : d 0.72 : e } t3 Omer { 0.875 : b 0.857 : c } t4 Tim { 0.9 : a 0.72 : d 0.718 : e } t5 Dan { 0.875 : b 0.857 : c 0.05 : d } t6 Bas { 0.875 : b 0.1 : f } Simplified probabilistic diagnosis database (adapted from [6]) Sebastiaan van Schaik
  12. 12. Seminar web data extraction > Introduction to uncertain data 9 / 26 Possible worlds D = {t1 , t2 , . . . , tn } (n transactions) tj = (p(j,1) , i1 ), . . . , (p(j,m) , im ) (m items in each transaction) D can be expanded to possible worlds: W = {W1 , . . . , W2nm }. Sebastiaan van Schaik
  13. 13. Seminar web data extraction > Introduction to uncertain data 9 / 26 Possible worlds D = {t1 , t2 , . . . , tn } (n transactions) tj = (p(j,1) , i1 ), . . . , (p(j,m) , im ) (m items in each transaction) D can be expanded to possible worlds: W = {W1 , . . . , W2nm }. Patient Diagnosis (including prob.) t1 Cheng { 0.9 : a 0.72 : d 0.718 : e 0.8 : f } t2 Andrey { 0.9 : a 0.81 : c 0.718 : d 0.72 : e } t3 Omer { 0.875 : b 0.857 : c } t4 Tim { 0.9 : a 0.72 : d 0.718 : e } t5 Dan { 0.875 : b 0.857 : c 0.05 : d } t6 Bas { 0.875 : b 0.1 : f } Pr[Wx ] = (1 − p(1,a) ) · p(1,d) · (1 − p(1,e) ) · p(1,f ) · p(2,a) · . . . · p(6,f ) = 0.1 · 0.72 · 0.29 · 0.2 · 0.9 · . . . · 0.1 ≈ 0.00000021 (one of the 218 possible worlds) Sebastiaan van Schaik
  14. 14. Seminar web data extraction > Mining uncertain data > Introduction 10 / 26 Introduction Approaches to mining frequent itemsets from uncertain data: U-Apriori[4] and p-Apriori[9] UF-growth[6] UFP-tree[1] ... Further focus: UF-growth: mining without candidate generation; p-Apriori: pruning using Chernoff bounds Sebastiaan van Schaik
  15. 15. Seminar web data extraction > Mining uncertain data > Introduction 11 / 26 Expected support Support of an itemset X turns into a random variable: E [sup(X )] = Pr[Wi ] · supWi (X ) Wi ∈W Sebastiaan van Schaik
  16. 16. Seminar web data extraction > Mining uncertain data > Introduction 11 / 26 Expected support Support of an itemset X turns into a random variable: E [sup(X )] = Pr[Wi ] · supWi (X ) Wi ∈W Enumerating all possible worlds is infeasible, however (because of independency assumptions): E [sup(X )] = Pr[x, tj ] tj ∈D x∈X (see [7, 6]) Sebastiaan van Schaik
  17. 17. Seminar web data extraction > Mining uncertain data > Introduction 12 / 26 Expected support (2) Patient Diagnosis (including prob.) t1 Cheng { 0.9 : a 0.72 : d 0.718 : e 0.8 : f } t2 Andrey { 0.9 : a 0.81 : c 0.718 : d 0.72 : e } t3 Omer { 0.875 : b 0.857 : c } t4 Tim { 0.9 : a 0.72 : d 0.718 : e } t5 Dan { 0.875 : b 0.857 : c 0.05 : d } t6 Bas { 0.875 : b 0.1 : f } Expected support of itemset X = {a, d} in patient diagnosis database: supWx (X ) = 2 E[sup(X )] = Pr[Wi ] · supWi (X ) Wi ∈W Sebastiaan van Schaik
  18. 18. Seminar web data extraction > Mining uncertain data > Introduction 12 / 26 Expected support (2) Patient Diagnosis (including prob.) t1 Cheng { 0.9 : a 0.72 : d 0.718 : e 0.8 : f } t2 Andrey { 0.9 : a 0.81 : c 0.718 : d 0.72 : e } t3 Omer { 0.875 : b 0.857 : c } t4 Tim { 0.9 : a 0.72 : d 0.718 : e } t5 Dan { 0.875 : b 0.857 : c 0.05 : d } t6 Bas { 0.875 : b 0.1 : f } Expected support of itemset X = {a, d} in patient diagnosis database: supWx (X ) = 2 E[sup(X )] = Pr[Wi ] · supWi (X ) Wi ∈W = Pr[x, tj ] tj ∈D x∈X = 0.9 · 0.72 + 0.9 · 0.71 + 0 · 0 + 0.9 · 0.72 + 0 · 0.05 + 0 · 0 = 1.935 Sebastiaan van Schaik
  19. 19. Seminar web data extraction > Mining uncertain data > Introduction 13 / 26 Frequent itemsets in probabilistic databases An itemset X is frequent iff: UF-growth: E[sup(X )] > minsup (also used in [4, 1] and many others) p-Apriori: Pr[sup(X ) > minsup] ≥ minprob Sebastiaan van Schaik
  20. 20. Seminar web data extraction > Mining uncertain data > UF-growth 14 / 26 Introduction to UF-growth Apriori versus UF-growth: Apriori-like algorithms generate and test candidate itemsets; UF-growth[6] (based on FP-growth[5]) grows a tree based on a probabilistic database. Sebastiaan van Schaik
  21. 21. Seminar web data extraction > Mining uncertain data > UF-growth 14 / 26 Introduction to UF-growth Apriori versus UF-growth: Apriori-like algorithms generate and test candidate itemsets; UF-growth[6] (based on FP-growth[5]) grows a tree based on a probabilistic database. Outline of procedure (example follows): 1 First scan: determine expected support of all items; 2 Second scan: create branch for each transaction (merging identical nodes when possible). Each node contains: An item; Its probability; Its occurrence count. Example: (a, 0.9, 2) Sebastiaan van Schaik
  22. 22. Seminar web data extraction > Mining uncertain data > UF-growth 14 / 26 Introduction to UF-growth Apriori versus UF-growth: Apriori-like algorithms generate and test candidate itemsets; UF-growth[6] (based on FP-growth[5]) grows a tree based on a probabilistic database. Outline of procedure (example follows): 1 First scan: determine expected support of all items; 2 Second scan: create branch for each transaction (merging identical nodes when possible). Each node contains: An item; Its probability; Its occurrence count. Example: (a, 0.9, 2) An itemset X is frequent iff: E[sup(X )] > minsup Sebastiaan van Schaik
  23. 23. Seminar web data extraction > Mining uncertain data > UF-growth 15 / 26 UF-tree example (1) Patient Diagnosis (including prob.) t1 Cheng { 0.9 : a 0.72 : d 0.718 : e 0.8 : f } t2 Andrey { 0.9 : a 0.81 : c 0.718 : d 0.72 : e } t3 Omer { 0.875 : b 0.857 : c } t4 Tim { 0.9 : a 0.72 : d 0.718 : e } t5 Dan { 0.875 : b 0.857 : c 0.05 : d } t6 Bas { 0.875 : b 0.1 : f } 1) determine exp. support 2) build tree E [sup({a})] = 2.7 E [sup({b})] = 2.625 E [sup({c})] = 2.524 E [sup({d})] = 2.20875 E [sup({e})] = 2.1575 E [sup({f })] = 0.9 (from [6]) Sebastiaan van Schaik
  24. 24. Seminar web data extraction > Mining uncertain data > UF-growth 16 / 26 UF-tree example (2) Extract frequent patterns from FP-tree: E [sup({a, e})] = 1 · 0.72 · 0.9 + 2 · 0.71875 · 0.9 = 1.94175 E [sup({c, e})] = 1 · 0.72 · 0.81 = 0.5832 E [sup({d, e})] = 1 · 0.72 · 0.71875 + 2 · 0.71875 · 0.72 = 1.5525 E [sup({a, d, e})] = 1 · 0.9 · 0.71875 · 0.72 + 2 · 0.9 · 0.72 · 0.71875 = 1.39725 Sebastiaan van Schaik
  25. 25. Seminar web data extraction > Mining uncertain data > UF-growth 17 / 26 UF-growth continued Mining larger itemsets can be done more efficiently using tree projections. Remarks: Nodes can only be merged when items have identical probabilities (otherwise, all occurence counts equal 1); Suggested solution in [6]: rounding of probabilities; Other solution (from [1]): store a carefully constructed summary of probabilities in each node. Might yield overestimation of expected support. Sebastiaan van Schaik
  26. 26. Seminar web data extraction > Mining uncertain data > p-Apriori 18 / 26 Introduction to p-Apriori Apriori has been extended to support uncertainty; New pruning techniques[9, 4, 3] improve efficiency; Note: the apriori (“downwards closure”) property still holds in the probabilistic case[1]; Goal: prune candidates, saving as much time as possible. Sebastiaan van Schaik
  27. 27. Seminar web data extraction > Mining uncertain data > p-Apriori 18 / 26 Introduction to p-Apriori Apriori has been extended to support uncertainty; New pruning techniques[9, 4, 3] improve efficiency; Note: the apriori (“downwards closure”) property still holds in the probabilistic case[1]; Goal: prune candidates, saving as much time as possible. In p-Apriori, an itemset X is frequent iff: Pr[sup(X ) > minsup] ≥ minprob Sebastiaan van Schaik
  28. 28. Seminar web data extraction > Mining uncertain data > p-Apriori 19 / 26 p-Apriori: advanced frequent itemset mining Sun et al. [9] use a simplified approach to modelling uncertainty: each tuple ti is associated with an existential probability pi . 1 Interesting course: Probability & Computing by James Worrell Sebastiaan van Schaik
  29. 29. Seminar web data extraction > Mining uncertain data > p-Apriori 19 / 26 p-Apriori: advanced frequent itemset mining Sun et al. [9] use a simplified approach to modelling uncertainty: each tuple ti is associated with an existential probability pi . In p-Apriori: itemset X is frequent if and only if: Pr[sup(X ) > minsup] ≥ minprob Let cnt(X ) denote the number of tuples containing X , then: cnt(X ) < minsup ⇒ X can not be frequent Chernoff bounds1 provide a strict bound on the tail distributions of sums of independent random variables. 1 Interesting course: Probability & Computing by James Worrell Sebastiaan van Schaik
  30. 30. Seminar web data extraction > Mining uncertain data > p-Apriori 20 / 26 p-Apriori: pruning using Chernoff Bounds (1) Each tuple ti is associated with an existential probability pi . Then: 1 with probability pi Yi = 0 with probability 1 − pi Y = Yi = sup(X ) Sebastiaan van Schaik
  31. 31. Seminar web data extraction > Mining uncertain data > p-Apriori 20 / 26 p-Apriori: pruning using Chernoff Bounds (1) Each tuple ti is associated with an existential probability pi . Then: 1 with probability pi Yi = 0 with probability 1 − pi Y = Yi = sup(X ) Furthermore: µ = E[sup(X )] minsup − µ − 1 δ = µ Pr[sup(X ) ≥ minsup] = Pr [sup(X ) > minsup − 1] = Pr [sup(X ) > (1 + δ)µ] Sebastiaan van Schaik
  32. 32. Seminar web data extraction > Mining uncertain data > p-Apriori 21 / 26 p-Apriori: pruning using Chernoff Bounds (2) Using a Chernoff bound (see [8], theorem 4.3 and exercise 4.1): 2−δµ if δ ≥ 2e − 1 Pr[sup(X ) ≥ minsup] < −δ 2 µ e 4 otherwise Therefore: an itemset X can not be frequent if: for δ ≥ 2e − 1 : 2−δµ < minprob −δ 2 µ for 0 < δ < 2e − 1 : e 4 < minprob Example with minprob = 0.4, minsup = 9 and E [sup(X )] = 3: 2 −δ 2 µ − 9−3−1 ( 3 ) ·3 25 e 4 =e 4 = e − 12 ≈ 0.125 < minprob Sebastiaan van Schaik
  33. 33. Seminar web data extraction > Mining uncertain data > p-Apriori 22 / 26 p-Apriori: finding frequent patterns (DP) The p-Apriori algorithm for finding frequent patterns resembles apriori: 1 Generate set of candidate k-itemsets Ck based on frequent itemsets of length k − 1 2 For each itemset X ∈ Ck : 1 Try pruning by using apriori property 2 Compute cnt(X ), try pruning using Chernoff bound 3 For each itemset X ∈ Ck left: compute pmf in O(n2 ) time, compare against minprob (association rules can be mined using the frequent patterns) Sebastiaan van Schaik
  34. 34. Seminar web data extraction > Summary & conclusion 23 / 26 Summary & conclusion Data mining of uncertain data is a new, fast moving field; Data uncertainty introduces a significant complexity layer; Different algorithms use different definitions and models; Algorithm performance greatly depends on data. Sebastiaan van Schaik
  35. 35. Seminar web data extraction > References 24 / 26 References I C. C. Aggarwal, Y. Li, and Jing Wang. Frequent pattern mining with uncertain data. discovery and data mining, pages 29–37, 2009. Jorge B. Bocca, Matthias Jarke, and Carlo Zaniolo, editors. Fast algorithms for mining association rules, volume 1215 of Proc 20th Int Conf Very Large Data Bases VLDB. Citeseer, 1994. C. K. Chui and B. Kao. decremental approach for mining frequent itemsets from uncertain data. Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining, pages 64–75, 2008. Sebastiaan van Schaik
  36. 36. Seminar web data extraction > References 25 / 26 References II C. K. Chui, Ben Kao, and Edward Hung. Mining frequent itemsets from uncertain data. Advances in Knowledge Discovery and Data Mining, pages 47–58, 2007. J. Han, J. Pei, Y. Yin, and R. Mao. Mining frequent patterns without candidate generation: A frequent-pattern tree approach. Data mining and knowledge discovery, 8(1):53–87, 2004. C. Leung, M. Mateo, and D. Brajczuk. tree-based approach for frequent pattern mining from uncertain data. Advances in Knowledge Discovery and Data Mining, pages 653–661, 2008. Sebastiaan van Schaik
  37. 37. Seminar web data extraction > References 26 / 26 References III C. K. S. Leung, B. Hao, and F. Jiang. Constrained frequent itemset mining from uncertain data streams. pages 120–127, 2010. R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, 1995. Liwen Sun, R. Cheng, and D. W. Cheung. Mining uncertain data with probabilistic guarantees. discovery and data mining, pages 273–282, 2010. Recommended by Dan Olteanu, read by Nov 12 4pm. Sebastiaan van Schaik

×