This document provides an introduction to association rule mining. It discusses how association rules are used to find frequent patterns and correlations among items in transactional databases. The document outlines the Apriori algorithm, which is the most influential algorithm for mining association rules. Apriori works in two steps: (1) finding all frequent itemsets whose support is above a minimum threshold, and (2) generating association rules from the frequent itemsets that have confidence above a minimum. An example of applying Apriori to a sample transactional database is provided to illustrate the algorithm.

1. Introduction to
Association Rules

2. Recap
Data Set Classification Model How?
We have seen four different types of approaches to classification :
• Decision trees (CART)
• Instance-based algorithms (kNN )
• Bayesian classifiers (Naïve Bayes)
• Neural Networks (Perceptron,ANN,CNN,SVM)

3. Agenda
 Introduction to Association Rules
 A Taxonomy of Association Rules
 Measures of Interest
 Apriori

4. Introduction to AR
 Ideas come from the market basket analysis (MBA)
 Let’s go shopping!
Milk, eggs, sugar,
bread
Eggs, sugar
Milk, eggs, cereal,
bread
Customer1
Customer2 Customer3
 What do my customer buy? Which product are bought together?
 Aim: Find associations and correlations between the different
items that customers place in their shopping basket

5. Introduction to AR
 Formalizing the problem a little bit
 Transaction Database T: a set of transactions T = {t1, t2, …, tn}
 Each transaction contains a set of items I (item set)
 An itemset is a collection of items I = {i1, i2, …, im}
 General aim:
 Find frequent/interesting patterns, associations, correlations, or
causal structures among sets of items or elements in
databases or other information repositories.
 Put this relationships in terms of association rules
 X  Y

6. Example of AR
Examples:
 bread  peanut-butter
 Cheese bread
TID Items
T1 bread, jelly, peanut-butter
T2 bread, peanut-butter
T3 bread, milk, peanut-butter
T4 Cheese, bread
T5 Cheese, milk
 Frequent itemsets: Items that frequently appear together
 I = {bread, peanut-butter}
 I = {Cheese, bread}

7. What’s an Interesting Rule?
 Support count (σ) TID Items
 Frequency of occurrence of
and itemset
 σ ({bread, peanut-butter}) = 3
 σ ({Cheese, bread}) = 1
 Support
T1
T2
T3
T4
T5
bread, jelly, peanut-butter
bread, peanut-butter
bread, milk, peanut-butter
Cheese bread
Cheese, milk
 Fraction of transactions that
contain an itemset
 s ({bread,peanut-butter}) = 3/5
 s ({Cheese, bread}) = 1/5
 Frequent itemset
 An itemset whose support is greater than or equal to a
minimum support threshold (minsup)

8. What’s an Interesting Rule?
 An association rule is an TID Items
implication of two itemsets
 X  Y
T1
T2
T3
T4
T5
bread, jelly, peanut-butter
bread, peanut-butter
bread, milk, peanut-butter
Cheese, bread
Cheese, milk
 Many measures of interest.
The two most used are:
 Support (s)
 The occurring frequency of the rule,
i.e., number of transactions that
contain both X and Y
s 
 (X Y)
# of trans.
 Confidence (c)
 The strength of the association,
i.e., measures of how often items in Y
appear in transactions that contain X
c 
 (X  Y )
(X)

9. Interestingness of Rules
TID s c
bread  peanut-butter 0.60 0.75
peanut-butter  bread 0.60 1.00
Cheese  bread 0.20 0.50
peanut-butter  jelly 0.20 0.33
jelly  peanut-butter 0.20 1.00
jelly  milk 0.00 0.00
TID Items
T1 bread, jelly, peanut-butter
T2 bread, peanut-butter
T3 bread, milk, peanut-butter
T4 Cheese, bread
T5 Cheese, milk
 Many other interesting measures
 The method presented herein are based on these two
approaches

10. Types of AR
 Binary association rules:
 bread  peanut-butter
 Quantitative association rules:
 weight in [70kg – 90kg]  height in [170cm – 190cm]
 Fuzzy association rules:
 weight in TALL  height in TALL
 Let’s start for the beginning
 Binary association rules – A priori

11. Apriori
 This is the most influential AR miner
 It consists of two steps
1. Generate all frequent itemsets whose support ≥ minsup
2. Use frequent itemsets to generate association rules
 So, let’s pay attention to the first step

12. Apriori
null
A B C D E
AB AD
AC AE BD
BC BE CE
CD DE
ABC ABE
ABD ACD ADE
ACE BCD BDE
BCE CDE
ABCD ABCE ABDE ACDE BCDE
ABCDE
Given d items, we have 2d possible itemsets.
 Do I have to generate them all?

13. Apriori
 Let’s avoid expanding all the graph
 Key idea:
 Downward closure property: Any subsets of a frequent itemset
are also frequent itemsets
 Therefore, the algorithm iteratively does:
 Create itemsets
 Only continue exploration of those whose support ≥ minsup

14. Example Itemset Generation
null
Infrequent
itemset
A B C D E
AB AD
AC AE BD
BC BE CE
CD DE
ABC ABE
ABD ACD ADE
ACE BCD BDE
BCE CDE
ABCD ABCE ABDE ACDE BCDE
ABCD
Given d items, we have 2d possible itemsets.
 Do I have to generate them all?

15. Recovering the Example
TID Items
T1 bread, jelly, peanut-butter
T2 bread, peanut-butter
T3 bread, milk, peanut-butter
T4 Cheese, bread
T5 Cheese, milk
Minimum support = 3
1-itemsets
Item count
2-itemsets
Item count
bread, peanut-b 3
ITEM COUNT
Bread 4
Peanut-butter 3
Jelly 1
Milk 1
Cheese 1

16. Apriori Algorithm
 k=1
 Generate frequent itemsets of length 1
 Repeat until no frequent itemsets are found
 k := k+1
 Generate itemsets of size k from the k-1 frequent itemsets
 Compute the support of each candidate by scanning DB

17. Example of Apriori Run
Itemset sup
Database TDB
C1
L1 {A} 2
{B} 3
1st scan
Itemset sup
{A} 2
{B} 3
{C} 3
{D} 1
{E} 3
{C} 3
{E} 3
C2 C2
Tid Items
10 A, C, D
20 B, C, E
30 A, B, C, E
40 B, E
Itemset
Itemset sup
L2 2nd scan {A, B}
{A, C}
{A, B} 1
{A, C} 2
{A, E}
{B, C}
{A, E}
{B, C}
1
2
{B, E}
{C, E}
3
2
Itemset sup
{A, C} 2
{B, C} 2
{B, E} 3
{C, E} 2
C3 L3
3rd scan
{B, E}
{C, E}
Itemset
{B, C, E}
Itemset sup
{B, C, E} 2

18. Apriori
 Remember that Apriori consists of two steps
1. Generate all frequent itemsets whose support ≥ minsup
2. Use frequent itemsets to generate association rules
 We accomplished step 1. So we have all frequent
itemsets
 So, let’s pay attention to the second step

19. Rule Generation in Apriori
 Given a frequent itemset L
 Find all non-empty subsets F in L, such that the association
rule F  {L-F} satisfies the minimum confidence
 Create the rule F  {L-F}
 If L={A,B,C}
 The candidate itemsets are: ABC, ACB, BCA, ABC,
BAC, CAB
 In general, there are 2K-2 candidate solutions, where k is the
length of the itemset L

20. Can you Be More Efficient?
 Can we apply the same trick used with support?
 Confidence does not have anti-monote property
 That is, c(ABD) > c(A D)?
 Don’t know!
 But confidence of rules generated from the same itemset
does have the anti-monote property
 L={A,B,C,D}
 C(ABCD) ≥ c(AB CD) ≥ c(A BCD)
 We can apply this property to prune the rule generation

21. Example of Efficient Rule Generation
ABCD
Low
confidence
ABCD ABDC ACDB BCDA
ABCD ACBD BCAD BDAD
ADBC CDAB
ABCD BACD CABD DABC

22. Challenges in AR Mining
 Challenges
 Apriori scans the data base multiple times
 Most often, there is a high number of candidates
 Support counting for candidates can be time expensive
 Several methods try to improve this points by
 Reduce the number of scans of the data base
 Shrink the number of candidates
 Counting the support of candidates more efficiently

23. MARKET BASKET ANALYSIS
• Technique used by large retailers to uncover associates between items.
• Combination of items that occurs together frequently in transaction
• To find frequently purchased item sets from large transactional Database.

24. • Cross selling
• Product placement
• Affinity promotion
• Fraud detection
• Customer Behavior
APPLICATIONS

25. REFRENCES
1.https://github.com/datacamp/Market-Basket-Analysis-in-python-live-training
2. https://youtu.be/4QIWJVVWJdQ
3. https://pbpython.com/market-basket-analysis.html