1. Data Mining: the Practice An Introduction Slides taken from : Data Mining by I. H. Witten and E. Frank

17. Concepts, instances, attributes Slides for Chapter 2 of Data Mining by I. H. Witten and E. Frank

34. The ARFF format % % ARFF file for weather data with some numeric features % @relation weather @attribute outlook {sunny, overcast, rainy} @attribute temperature numeric @attribute humidity numeric @attribute windy {true, false} @attribute play? {yes, no} @data sunny, 85, 85, false, no sunny, 80, 90, true, no overcast, 83, 86, false, yes ...

41. Output: representing structural patterns

52. The contact lenses data None Reduced Yes Hypermetrope Pre-presbyopic None Normal Yes Hypermetrope Pre-presbyopic None Reduced No Myope Presbyopic None Normal No Myope Presbyopic None Reduced Yes Myope Presbyopic Hard Normal Yes Myope Presbyopic None Reduced No Hypermetrope Presbyopic Soft Normal No Hypermetrope Presbyopic None Reduced Yes Hypermetrope Presbyopic None Normal Yes Hypermetrope Presbyopic Soft Normal No Hypermetrope Pre-presbyopic None Reduced No Hypermetrope Pre-presbyopic Hard Normal Yes Myope Pre-presbyopic None Reduced Yes Myope Pre-presbyopic Soft Normal No Myope Pre-presbyopic None Reduced No Myope Pre-presbyopic hard Normal Yes Hypermetrope Young None Reduced Yes Hypermetrope Young Soft Normal No Hypermetrope Young None Reduced No Hypermetrope Young Hard Normal Yes Myope Young None Reduced Yes Myope Young Soft Normal No Myope Young None Reduced No Myope Young Recommended lenses Tear production rate Astigmatism Spectacle prescription Age

53. A complete and correct rule set If tear production rate = reduced then recommendation = none If age = young and astigmatic = no and tear production rate = normal then recommendation = soft If age = pre-presbyopic and astigmatic = no and tear production rate = normal then recommendation = soft If age = presbyopic and spectacle prescription = myope and astigmatic = no then recommendation = none If spectacle prescription = hypermetrope and astigmatic = no and tear production rate = normal then recommendation = soft If spectacle prescription = myope and astigmatic = yes and tear production rate = normal then recommendation = hard If age young and astigmatic = yes and tear production rate = normal then recommendation = hard If age = pre-presbyopic and spectacle prescription = hypermetrope and astigmatic = yes then recommendation = none If age = presbyopic and spectacle prescription = hypermetrope and astigmatic = yes then recommendation = none

55. A decision tree for this problem

57. Linear regression for the CPU data PRP = -56.1 + 0.049 MYCT + 0.015 MMIN + 0.006 MMAX + 0.630 CACH - 0.270 CHMIN + 1.46 CHMAX

59. Regression tree for the CPU data

60. Model tree for the CPU data

63. Representing clusters I Simple 2-D representation Venn diagram Overlapping clusters

64. Representing clusters II 1 2 3 a 0.4 0.1 0.5 b 0.1 0.8 0.1 c 0.3 0.3 0.4 d 0.1 0.1 0.8 e 0.4 0.2 0.4 f 0.1 0.4 0.5 g 0.7 0.2 0.1 h 0.5 0.4 0.1 … Probabilistic assignment Dendrogram NB: dendron is the Greek word for tree

68. Evaluating the weather attributes * indicates a tie 3/6 True  No* 5/14 2/8 False  Yes Windy 1/7 Normal  Yes 4/14 3/7 High  No Humidity 5/14 4/14 Total errors 1/4 Cool  Yes 2/6 Mild  Yes 2/4 Hot  No* Temp 2/5 Rainy  Yes 0/4 Overcast  Yes 2/5 Sunny  No Outlook Errors Rules Attribute No True High Mild Rainy Yes False Normal Hot Overcast Yes True High Mild Overcast Yes True Normal Mild Sunny Yes False Normal Mild Rainy Yes False Normal Cool Sunny No False High Mild Sunny Yes True Normal Cool Overcast No True Normal Cool Rainy Yes False Normal Cool Rainy Yes False High Mild Rainy Yes False High Hot Overcast No True High Hot Sunny No False High Hot Sunny Play Windy Humidity Temp Outlook

70. Which attribute to select?

71. Which attribute to select?

76. Continuing to split gain( Temperature ) = 0.571 bits gain( Humidity ) = 0.971 bits gain( Windy ) = 0.020 bits

88. Pseudo-code for PRISM For each class C Initialize E to the instance set While E contains instances in class C Create a rule R with an empty left-hand side that predicts class C Until R is perfect (or there are no more attributes to use) do For each attribute A not mentioned in R, and each value v, Consider adding the condition A = v to the left-hand side of R Select A and v to maximize the accuracy p/t (break ties by choosing the condition with the largest p) Add A = v to R Remove the instances covered by R from E

136. A hypothetical lift chart 40% of responses for 10% of cost 80% of responses for 40% of cost

143. Summary of some measures Explanation Plot Domain TP/(TP+FN) TP/(TP+FP) Recall Precision Information retrieval Recall-precision curve TP/(TP+FN) FP/(FP+TN) TP rate FP rate Communications ROC curve TP (TP+FP)/(TP+FP+TN+FN) TP Subset size Marketing Lift chart

145. Cost curves: example with costs