Data Mining.ppt

  • 5,572 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
5,572
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
236
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Data Mining-Knowledge Presentation 2 Prof. Sin-Min Lee Lecture 26
  • 2. Overview
    • Association rules are useful in that they suggest hypotheses for future research
    • Association rules integrated into the generic actual argument model can assist in identifying the most plausible claim from given data items in a forward inference way or the likelihood of missing data values in a backward inference way
  • 3.
    • What is data mining ? What is knowledge discovery from databases KDD?
    • knowledge discovery in databases (KDD) is the 'non trivial extraction of nontrivial of implicit, previously unknown, and potentially useful information from data
  • 4.
    • KDD encompasses a number of different technical approaches, such as clustering, data summarization, learning classification rules, finding dependency networks, analyzing changes, and detecting anomalies
    • KDD has only recently emerged because we only recently have been gathering vast quantities of data
  • 5.
    • Examples of KDD studies
    • Mangasarian et al (1997) Breast Cancer diagnosis. A sample from breast lump mass is assessed by:
        • mammagrophy (not sensitive 68%-79%)
        • data mining from FNA test results and visual inspection (65%-98%)
        • surgery (100% but invasive, expensive)
    • Basket analysis. People who buy nappies also buy beer
    • NBA. National Basketball Association of America. Player pattern profile. Bhandary et al (1997)
    • Credit card fraud detection
    • Stranieri/Zeleznikow (1997) predict family law property outcomes
    • Rissland and Friedman (1997) discovers a change in the concept of ‘good faith’ in US Bankruptcy cases
    • Pannu (1995) discovers a prototypical case from a library of cases
    • Wilkins and Pillaipakkamnatt (1997) predicts the time a case takes to be heard
    • Veliev et al (1999) association rules for economic analaysis
  • 6.
    • Overview of process of knowledge discovery in databases ?
    Interpret patterns Data mining Trans form Pre process Select knowledge patterns Transform ed data Pre-proces sed data Target data Raw data from Fayyad, Pitatetsky-Shapiro, Smyth (1996)
  • 7.
    • Phase 4. Data mining
    • Finding patterns in data or fitting models to data
    • Categories of techniques
      • Predictive (classification: neural networks, rule induction, linear, multiple regression)
      • Segmentation (clustering, k-means, k-median)
      • Summarisation (associations, visualisation)
      • Change detection/modelling
  • 8. What Is Association Mining?
    • Association rule mining:
      • Finding frequent patterns, associations, correlations, or causal structures among sets of items or objects in transaction databases, relational databases, and other information repositories.
    • Applications:
      • Basket data analysis, cross-marketing, catalog design, loss-leader analysis, clustering, classification, etc.
    • Examples.
      • Rule form: “ Body  ead [support, confidence]”.
      • buys(x, “diapers”)  buys(x, “beers”) [0.5%, 60%]
      • major(x, “CS”) ^ takes(x, “DB”)  grade(x, “A”) [1%, 75%]
  • 9. More examples
      • age(X, “20..29”) ^ income(X, “20..29K”)  buys(X, “PC”) [support = 2%, confidence = 60%]
      • contains(T, “computer”)  contains(x, “software”) [1%, 75%]
  • 10.
    • Association rules are a data mining technique
    • An association rules tell us something about the association between two attributes
    • Agrawal et al (1993) developed the first association rule algorithm, Apriori
    • A famous (but unsubstantiated AR) from a hypothetical supermarket transaction database is if nappies then beer (80%) Read this as nappies are bought implies beer are bought 80% of the time
    • Association rules have only recently been applied to law with promising results
    • Association rules can automatically discover rules that may prompt an analyst to think of hypothesis they would otherwise have considered
  • 11. Rule Measures: Support and Confidence
    • Find all the rules X & Y  Z with minimum confidence and support
      • support , s , probability that a transaction contains {X  Y  Z}
      • confidence , c , conditional probability that a transaction having {X  Y} also contains Z
    • Let minimum support 50%, and minimum confidence 50%, we have
      • A  C (50%, 66.6%)
      • C  A (50%, 100%)
    Customer buys diaper Customer buys both Customer buys beer Support and confidence are two independent notions.
  • 12. Mining Association Rules—An Example
    • For rule A  C :
      • support = support({ A  C }) = 50%
      • confidence = support({ A  C })/support({ A }) = 66.6%
    Min. support 50% Min. confidence 50%
  • 13. Two Step Association Rule Mining
    • Step 1: Frequent itemset generation – use Support
    • Step 2: Rule generation – use Confidence
  • 14. {milk, bread} is a frequent item set. Folks buying milk, also buy bread. Is it also true?: “Folks buying bread also buy milk.”
  • 15.
    • Confidence and support of an association rule
    • 80% is the confidence of the rule if nappies then beer (80%). This is calculated by n2/n1 where:
      • n1 = no of records where nappies are bought
      • n2 = no of records where nappies were bought and beer was also bought.
    • if 1000 transactions for nappies, and of those, 800 also had beer then confidence is 80%.
    • A rule may have a high confidence but not be interesting because it doesn’t apply to many records in the database. i.e. no. of records where nappies were bought with beer / total records.
    • Rules that may be interesting have a confidence level and support level above a user set threshold
  • 16.
    • Interesting rules: Confidence and support of an association rule
    • if 1000 transactions for nappies, and of those, 800 also had beer then confidence is 80%.
    • A rule may have a high confidence but not be interesting because it doesn’t apply to many records in the database. i.e. no. of records where nappies were bought with beer / total records.
    • Rules that may be interesting have a confidence level and support level above a user set threshold
  • 17. Association rule screen shot with A-Miner from Split Up data set
    • In 73.4% of cases where the wife's needs are some to high then the husband's future needs are few to some.
    • Prompts an analyst to posit plausible hypothesis e.g. it may be the case that the rule reflects the fact that more women remain custodial parents of the children following divorce than men do. The women that have some to high needs may do so because of their obligation to children.
  • 18. Mining Frequent Itemsets: the Key Step
    • Find the frequent itemsets : the sets of items that have minimum support
      • A subset of a frequent itemset must also be a frequent itemset – Apriori principle
        • i.e., if { AB } is a frequent itemset, both { A } and { B } should be a frequent itemset
      • Iteratively find frequent itemsets with cardinality from 1 to k (k- itemset )
    • Use the frequent itemsets to generate association rules.
  • 19. The Apriori Algorithm
    • Join Step : C k is generated by joining L k-1 with itself
    • Prune Step : Any (k-1)-itemset that is not frequent cannot be a subset of a frequent k-itemset
    • Pseudo-code :
        • C k : Candidate itemset of size k
        • L k : frequent itemset of size k
        • L 1 = {frequent items};
        • for ( k = 1; L k !=  ; k ++) do begin
        • C k+1 = candidates generated from L k ;
        • for each transaction t in database do
          • increment the count of all candidates in C k+1 that are contained in t
        • L k+1 = candidates in C k+1 with min_support
        • end
        • return  k L k ;
  • 20. Association rules in law
    • Association rules generators are typically packaged with very expensive data mining suites. We developed A-Miner (available from authors) for a PC platform.
    • Typically, too many association rules are generated for feasible analysis. So, our current research involves exploring metrics of interesting to restrict numbers of rules that might be interesting
    • In general, structured data is not collected in law as it is in other domains so very large databases are rare
    • Our current research involves 380,000 records from a Legal Aid organization data base that contains data on client features.
    • ArgumentDeveloper shell that can be used by judges to structure their reasoning in a way that will facilitate data collection and reasoning
  • 21. The Apriori Algorithm — Example Database D Scan D C 1 L 1 L 2 C 2 C 2 Scan D Support = 2
  • 22. Join Operation — Example C 3 L 3 Scan D L 2 L 2 {1 3} {1 3} {1 3} {2 3} {1 3} {2 5} {1 3} {3 5} null {1 2 3} null {1 3 5} {2 3} {2 3} {2 3} {2 5} {2 3} {3 5} null {2 3 5} {2 3 5} {1 2} {1 5} Infrequent Subset {2 5} {2 5} {2 5} {3 5} null {2 3 5} L 2 L 2 join
  • 23. Anti-Monotone Property If a set cannot pass a test, all of its supersets will fail the same test as well. If {2 3} does not have a support, nor will {1 2 3}, {2 3 5}, {1 2 3 5}, etc. If {2 3} occurs only in 5 times, can {2 3 5} occur in 8 times?
  • 24. How to Generate Candidates?
    • Suppose the items in L k-1 are listed in an order
    • Step 1: self-joining L k-1
      • insert into C k
      • select p.item 1 , p.item 2 , …, p.item k-1 , q.item k-1
      • from L k-1 p, L k-1 q
      • where p.item 1 =q.item 1 , …, p.item k-2 =q.item k-2 , p.item k-1 < q.item k-1
    • Step 2: pruning
      • forall itemsets c in C k do
        • forall (k-1)-subsets s of c do
          • if (s is not in L k-1 ) then delete c from C k
  • 25. Example of Generating Candidates
    • L 3 = { abc, abd, acd, ace, bcd }
    • Self-joining: L 3 *L 3
      • abcd from abc and abd
      • acde from acd and ace
    • Pruning:
      • acde is removed because ade is not in L 3
    • C 4 ={ abcd }
    Problem of generate-&-test heuristic
  • 26. Association rules can be used for forward and backward inferences in the generic/actual argument model for sentencing armed robbery
  • 27. Generic/actual argument model for sentencing armed robbery
  • 28. Forward inference: confidence
    • In the sentence actual argument database the following outcomes were noted for the inputs suggested:
    57% 0.1% 0% 12% 2% 10% 16% 0% 0% 0%
  • 29. If extremely serious pattern of priors then imprisonment If very serious pattern of priors then imprisonment If serious pattern of priors then imprisonment If not so serious pattern of priors then imprisonment If no prior convictions then imprisonment Backward inference: constructing the strongest argument If all the items you suggest AND 90% 2% 75% 7% 68% 17% 78% 17% 2% 3%
  • 30. Conclusion
    • Data mining or Knowledge discovery from databases has not been appropriately exploited in law to date.
    • Association rules are useful in that they suggest hypotheses for future research
    • Association rules integrated into the generic actual argument model can assist in identifying the most plausible claim from given data items in a forward inference way or the likelihood of missing data values in a backward inference way
  • 31. Generating Association Rules
    • For each nonempty subset s of l , output the rule:
    • s => (l - s)
    • if support_count(l) / support_count(s) >= min_conf
    • where min_conf is the minimum confidence threshold.
    l = {2 3 5}, {2 3}, {2}, {3 5}, {3}, {2 5}, & {5}. {2 3} => {5} {3 5} => {2} {2 5} => {3} {2} => {3 5} {3} => {2 5} {5} => {2 3} s of l are Candidate rules:
  • 32. Generating Association Rules
    • if support_count(l) / support_count(s) >= min_conf (e.g,75%),
    • then introduce the rule s => (l - s).
    l = {2 3 5} s = {2 3} {2} {3 5} {3} {2 5} {5} {2 3} => {5} : 2/2 {3 5} => {2} : 2/2 {2 5} => {3} : 2/3 {2} => {3 5} : 2/3 {3} => {2 5} : 2/3 {5} => {2 3} : 2/3
  • 33. Presentation of Association Rules (Table Form )
  • 34. Visualization of Association Rule Using Plane Graph
  • 35. Visualization of Association Rule Using Rule Graph
  • 36.  
  • 37.  
  • 38.
    • Decision tree is a classifier in the form of a tree structure where each node is either:  
    •        a leaf node , indicating a class of instances, or
    •        a decision node that specifies some test to be carried out on a single attribute   value, with one branch and sub-tree for each possible outcome of the test.
    •  
    • A decision tree can be used to classify an instance by starting at the root of the tree and moving through it until a leaf node, which provides the classification of the instance.
    •  
  • 39.
    • Example: Decision making in the London stock market
    •  
    • Suppose that the major factors affecting the London stock market are:
    •  
    •           what it did yesterday;
    •           what the New York market is doing today;
    •           bank interest rate;
    •           unemployment rate;
    •           England’s prospect at cricket.
    •  
  • 40.  
  • 41.  
  • 42. The process of predicting an instance by this decision tree can also be expressed by answering the questions in the following order: Is unemployment high? YES: The London market will rise today NO: Is the New York market rising today? YES: The London market will rise today NO: The London market will not rise today.  
  • 43.
    • Decision tree induction is a typical inductive approach to learn knowledge on classification. The key requirements to do mining with decision trees are:
    •        Attribute-value description : object or case must be expressible in terms of a fixed collection of properties or attributes.
    •         Predefined classes : The categories to which cases are to be assigned must have been established beforehand (supervised data).
    •        Discrete classes : A case does or does not belong to a particular class, and there must be for more cases than classes.
    •        Sufficient data : Usually hundreds or even thousands of training cases.
    •         “ Logical” classification model : Classifier that can be only expressed as decision trees or set of production rules
    •  
    •  
  • 44.  
  • 45.  
  • 46. An appeal of market analysis comes from the clarity and utility of its results, which are in the form of association rules . There is an intuitive appeal to a market analysis because it expresses how tangible products and services relate to each other, how they tend to group together. A rule like, “if a customer purchases three way calling, then that customer will also purchase call waiting” is clear. Even better, it suggests a specific course of action, like bundling three-way calling with call waiting into a single service package. While association rules are easy to understand, they are not always useful.
  • 47.
    • The following three rules are examples of real rules generated from real data:
    •  On Thursdays, grocery store consumers often purchase diapers and beer together.
    •      Customers who purchase maintenance agreements are very likely to purchase large appliances.
    •     When a new hardware store opens, one of the most commonly sold items is toilet rings.
    • These three examples illustrate the three common types of rules produced by association rule analysis: the useful , the trivial , and the inexplicable .
  • 48.  
  • 49. OLAP (Summarization) Display Using MS/Excel 2000
  • 50. Market-Basket-Analysis (Association)—Ball graph
  • 51. Display of Association Rules in Rule Plane Form
  • 52. Display of Decision Tree (Classification Results)
  • 53. Display of Clustering (Segmentation) Results
  • 54. 3D Cube Browser