Outline
1. Data Mining (DM) ~ KDD [Definition]
2. DM Technique
-> Association rules [support & confidence]
3. Example
(4. ...
1. Data Mining ~ KDD [Definition]
- "Data mining (DM), also called KnowledgeDiscovery in Databases (KDD), is the process
o...
1. Data Mining ~ KDD [Definition]
Data Mining techniques
•
•
•
•
•
•

Information Visualization
k-nearest neighbor
decisio...
2. Association rules
Support
Every association rule has a support and a confidence.
“The support is the percentage of tran...
2. Association rules
Support

An itemset is called frequent if its support is equal or
greater than an agreed upon minimal...
2. Association rules
Confidence
Every association rule has a support and a confidence.
An association rule is of the form:...
2. Association rules
Confidence

We should only consider rules derived from
itemsets with high support, and that also have...
3. Example
Example: Database with transactions ( customer_# : item_a1, item_a2, … )

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

3, 5,...
3. Example
Example: Database with transactions ( customer_# : item_a1, item_a2, … )

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

3, 5,...
3. Example
Example: Database with transactions ( customer_# : item_a1, item_a2, … )

Conf ( {5} => {8} ) ? 80% Done.
Conf ...
3. Example
Example: Database with transactions ( customer_# : item_a1, item_a2, … )

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

3, 5,...
3. Example
Example: Database with transactions ( customer_# : item_a1, item_a2, … )

Conf( {9} => {3} ) = 100%. Done.
Noti...
Apriori Algorithm
• In computer science and data mining, Apriori is
a classic algorithm for learning association rules.
• ...
Definition (contd.)
• Apriori uses a "bottom up" approach, where
frequent subsets are extended one item at a
time (a step ...
15
Steps to Perform Apriori
Algorithm

16
Apriori Algorithm Examples
Problem Decomposition
Transaction ID Items Bought
1
Shoes, Shirt, Jacket
2
Shoes,Jacket
3
Shoes...
The Apriori Algorithm — Example
Min support =50%

Database D
TID
100
200
300
400

itemset sup.
C1
{1}
2
{2}
3
Scan D
{3}
3...
Pseudo Code for Apriori
Algorithm

19
Apriori
Advantages/Disadvantages
• Advantages
– Uses large itemset property
– Easily parallelized
– Easy to implement

• D...
Summary
•
•
•
•
•
•

Association Rules form an very applied data mining
approach.
Association Rules are derived from frequ...
Upcoming SlideShare
Loading in...5
×

Rmining

113

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
113
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Rmining

  1. 1. Outline 1. Data Mining (DM) ~ KDD [Definition] 2. DM Technique -> Association rules [support & confidence] 3. Example (4. Apriori Algorithm)
  2. 2. 1. Data Mining ~ KDD [Definition] - "Data mining (DM), also called KnowledgeDiscovery in Databases (KDD), is the process of automatically searching large volumes of data for patterns using specific DM technique." - [more formal definition] KDD ~ "the non-trivial extraction of implicit, previously unknown and potentially useful knowledge from data"
  3. 3. 1. Data Mining ~ KDD [Definition] Data Mining techniques • • • • • • Information Visualization k-nearest neighbor decision trees neural networks association rules …
  4. 4. 2. Association rules Support Every association rule has a support and a confidence. “The support is the percentage of transactions that demonstrate the rule.” Example: Database with transactions ( customer_# : item_a1, item_a2, …) 1: 2: 3: 4: 1, 3, 5. 1, 8, 14, 17, 12. 4, 6, 8, 12, 9, 104. 2, 1, 8. support {8,12} = 2 (,or 50% ~ 2 of 4 customers) support {1, 5} = 1 (,or 25% ~ 1 of 4 customers ) support {1} = 3 (,or 75% ~ 3 of 4 customers)
  5. 5. 2. Association rules Support An itemset is called frequent if its support is equal or greater than an agreed upon minimal value – the support threshold add to previous example: if threshold 50% then itemsets {8,12} and {1} called frequent
  6. 6. 2. Association rules Confidence Every association rule has a support and a confidence. An association rule is of the form: X => Y • X => Y: if someone buys X, he also buys Y The confidence is the conditional probability that, given X present in a transition , Y will also be present. Confidence measure, by definition: Confidence(X=>Y) equals support(X,Y) / support(X)
  7. 7. 2. Association rules Confidence We should only consider rules derived from itemsets with high support, and that also have high confidence. “A rule with low confidence is not meaningful.” Rules don’t explain anything, they just point out hard facts in data volumes.
  8. 8. 3. Example Example: Database with transactions ( customer_# : item_a1, item_a2, … ) 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 3, 5, 8. 2, 6, 8. 1, 4, 7, 10. 3, 8, 10. 2, 5, 8. 1, 5, 6. 4, 5, 6, 8. 2, 3, 4. 1, 5, 7, 8. 3, 8, 9, 10. Conf ( {5} => {8} ) ? supp({5}) = 5 , supp({8}) = 7 , supp({5,8}) = 4, then conf( {5} => {8} ) = 4/5 = 0.8 or 80%
  9. 9. 3. Example Example: Database with transactions ( customer_# : item_a1, item_a2, … ) 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 3, 5, 8. 2, 6, 8. 1, 4, 7, 10. 3, 8, 10. 2, 5, 8. 1, 5, 6. 4, 5, 6, 8. 2, 3, 4. 1, 5, 7, 8. 3, 8, 9, 10. Conf ( {5} => {8} ) ? 80% Done. Conf ( {8} => {5} ) ? supp({5}) = 5 , supp({8}) = 7 , supp({5,8}) = 4, then conf( {8} => {5} ) = 4/7 = 0.57 or 57%
  10. 10. 3. Example Example: Database with transactions ( customer_# : item_a1, item_a2, … ) Conf ( {5} => {8} ) ? 80% Done. Conf ( {8} => {5} ) ? 57% Done. Rule ( {5} => {8} ) more meaningful then Rule ( {8} => {5} )
  11. 11. 3. Example Example: Database with transactions ( customer_# : item_a1, item_a2, … ) 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 3, 5, 8. 2, 6, 8. 1, 4, 7, 10. 3, 8, 10. 2, 5, 8. 1, 5, 6. 4, 5, 6, 8. 2, 3, 4. 1, 5, 7, 8. 3, 8, 9, 10. Conf ( {9} => {3} ) ? supp({9}) = 1 , supp({3}) = 1 , supp({3,9}) = 1, then conf( {9} => {3} ) = 1/1 = 1.0 or 100%. OK?
  12. 12. 3. Example Example: Database with transactions ( customer_# : item_a1, item_a2, … ) Conf( {9} => {3} ) = 100%. Done. Notice: High Confidence, Low Support. -> Rule ( {9} => {3} ) not meaningful
  13. 13. Apriori Algorithm • In computer science and data mining, Apriori is a classic algorithm for learning association rules. • Apriori is designed to operate on databases containing transactions (for example, collections of items bought by customers, or details of a website frequentation). • The algorithm attempts to find subsets which are common to at least a minimum number C (the cutoff, or confidence threshold) of the itemsets. 13
  14. 14. Definition (contd.) • Apriori uses a "bottom up" approach, where frequent subsets are extended one item at a time (a step known as candidate generation, and groups of candidates are tested against the data. • The algorithm terminates when no further successful extensions are found. • Apriori uses breadth-first search and a hash tree structure to count candidate item sets efficiently. 14
  15. 15. 15
  16. 16. Steps to Perform Apriori Algorithm 16
  17. 17. Apriori Algorithm Examples Problem Decomposition Transaction ID Items Bought 1 Shoes, Shirt, Jacket 2 Shoes,Jacket 3 Shoes, Jeans 4 Shirt, Sweatshirt If the minimum support is 50%, then {Shoes, Jacket} is the only 2itemset that satisfies the minimum support. Frequent Itemset {Shoes} {Shirt} {Jacket} {Shoes, Jacket} Support 75% 50% 50% 50% If the minimum confidence is 50%, then the only two rules generated from this 2itemset, that have confidence greater than 50%, are: Shoes ⇒ Jacket Support=50%, Confidence=66% Jacket ⇒ Shoes Support=50%, Confidence=100% 17
  18. 18. The Apriori Algorithm — Example Min support =50% Database D TID 100 200 300 400 itemset sup. C1 {1} 2 {2} 3 Scan D {3} 3 {4} 1 {5} 3 Items 134 235 1235 25 L2 itemset sup C2 itemset sup 2 2 3 2 {1 {1 {1 {2 {2 {3 C3 itemset {2 3 5} Scan D {1 3} {2 3} {2 5} {3 5} 2} 3} 5} 3} 5} 5} 1 2 1 2 3 2 L1 itemset sup. {1} {2} {3} {5} 2 3 3 3 C2 itemset {1 2} Scan D L3 itemset sup {2 3 5} 2 {1 {1 {2 {2 {3 3} 5} 3} 5} 5} 18
  19. 19. Pseudo Code for Apriori Algorithm 19
  20. 20. Apriori Advantages/Disadvantages • Advantages – Uses large itemset property – Easily parallelized – Easy to implement • Disadvantages – Assumes transaction database is memory resident. – Requires many database scans. 20
  21. 21. Summary • • • • • • Association Rules form an very applied data mining approach. Association Rules are derived from frequent itemsets. The Apriori algorithm is an efficient algorithm for finding all frequent itemsets. The Apriori algorithm implements level-wise search using frequent item property. The Apriori algorithm can be additionally optimized. There are many measures for association rules. 21
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×