Roger Barga, profile picture
Uploaded byRoger Barga
PDF, PPTX375 views

Barga Data Science lecture 7

This document appears to be lecture slides for a course on deriving knowledge from data at scale. It covers many topics related to building machine learning models including data preparation, feature selection, classification algorithms like decision trees and support vector machines, and model evaluation. It provides examples applying these techniques to a Titanic passenger dataset to predict survival. It emphasizes the importance of data wrangling and discusses various feature selection methods.

Embed presentation

Download as PDF, PPTX
1 / 143
2 / 143
3 / 143
4 / 143
5 / 143
6 / 143
7 / 143
8 / 143
9 / 143
10 / 143
11 / 143
12 / 143
13 / 143
14 / 143
15 / 143
16 / 143
17 / 143
18 / 143
19 / 143
20 / 143
21 / 143
22 / 143
23 / 143
24 / 143
25 / 143
26 / 143
27 / 143
28 / 143
29 / 143
30 / 143
31 / 143
32 / 143
33 / 143
34 / 143
35 / 143
36 / 143
37 / 143
38 / 143
39 / 143
40 / 143
41 / 143
42 / 143
43 / 143
44 / 143
45 / 143
46 / 143
47 / 143
48 / 143
49 / 143
50 / 143
51 / 143
52 / 143
53 / 143
54 / 143
55 / 143
56 / 143
57 / 143
58 / 143
59 / 143
60 / 143
61 / 143
62 / 143
63 / 143
64 / 143
65 / 143
66 / 143
67 / 143
68 / 143
69 / 143
70 / 143
71 / 143
72 / 143
73 / 143
74 / 143
75 / 143
76 / 143
77 / 143
78 / 143
79 / 143
80 / 143
81 / 143
82 / 143
83 / 143
84 / 143
85 / 143
86 / 143
87 / 143
88 / 143
89 / 143
90 / 143
91 / 143
92 / 143
93 / 143
94 / 143
95 / 143
96 / 143
97 / 143
98 / 143
99 / 143
100 / 143
101 / 143
102 / 143
103 / 143
104 / 143
105 / 143
106 / 143
107 / 143
108 / 143
109 / 143
110 / 143
111 / 143
112 / 143
113 / 143
114 / 143
115 / 143
116 / 143
117 / 143
118 / 143
119 / 143
120 / 143
121 / 143
122 / 143
123 / 143
124 / 143
125 / 143
126 / 143
127 / 143
128 / 143
129 / 143
130 / 143
131 / 143
132 / 143
133 / 143
134 / 143
135 / 143
136 / 143
137 / 143
138 / 143
139 / 143
140 / 143
141 / 143
142 / 143
143 / 143
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Good data preparation is key to producing valid and reliable modelsโ€ฆ
Deriving Knowledge from Data at Scale Lecture 7 Agenda
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Heads Up, Lecture 8 Agenda
Deriving Knowledge from Data at Scale Course Project
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale https://www.kaggle.com/c/springleaf-marketing-response not Determine whether to send a direct mail piece to a customer
Deriving Knowledge from Data at Scale The Data
Deriving Knowledge from Data at Scale The Rules
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale what is the data telling you
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale The Tragedy of the Titanic
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Does the new data point x* exactly match a previous point xi? If so, assign it to the same class as xi Otherwise, just guess. This is the โ€œroteโ€ classifier
Deriving Knowledge from Data at Scale Does the new data point x* match a set pf previous points xi on some specific attribute? If so, take a vote to determine class. Example: If most females survived, then assume every female survives But there are lots of possible rules like this. And an attribute can have more than two values. If most people under 4 years old survive, then assume everyone under 4 survives If most people with 1 sibling survive, then assume everyone with 1 sibling survives How do we choose?
Deriving Knowledge from Data at Scale IF sex=โ€˜femaleโ€™ THEN survive=yes ELSE IF sex=โ€˜maleโ€™ THEN survive = no confusion matrix no yes <-- classified as 468 109 | no 81 233 | yes (468 + 233) / (468+109+81+233) = 79% correct (and 21% incorrect) Not bad!
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale IF pclass=โ€˜1โ€™ THEN survive=yes ELSE IF pclass=โ€˜2โ€™ THEN survive=yes ELSE IF pclass=โ€˜3โ€™ THEN survive=no confusion matrix no yes <-- classified as 372 119 | no 177 223 | yes (372 + 223) / (372+119+223+177) = 67% correct (and 33% incorrect) a little worse
Deriving Knowledge from Data at Scale Support Vector Machines
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale fx yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data? Estimation: w: weight vector x: data vector
Deriving Knowledge from Data at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?
Deriving Knowledge from Data at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?
Deriving Knowledge from Data at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?
Deriving Knowledge from Data at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Any of these would be fine.. ..but which is best?
Deriving Knowledge from Data at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Define the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.
Deriving Knowledge from Data at Scale O X O O O X X X SUPPORT VECTORS
Deriving Knowledge from Data at Scale O X X O O O X X X SUPPORT VECTORS
Deriving Knowledge from Data at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM) Linear SVM
Deriving Knowledge from Data at Scale2016/4/12 37 fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x + b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM)Support Vectors are those datapoints that the margin pushes up against Linear SVM
Deriving Knowledge from Data at Scale denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Support Vectors are those datapoints that the margin pushes up against 1. Intuitively this feels safest. 2. If weโ€™ve made a small error in the location of the boundary this gives us least chance of causing a misclassification. 3. LOOCV (leave one out cross validation) is easy since the model is immune to removal of any nonsupport-vector data points. 5. Empirically it works very well.
Deriving Knowledge from Data at Scale Class 1 Class 2 m
Deriving Knowledge from Data at Scale This is going to be a problem!
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale What if the data looks like that below?... kernel function
Deriving Knowledge from Data at Scale What if the data looks like that below?...
Deriving Knowledge from Data at Scale Everything you need to know in one slideโ€ฆ
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Probably the most tricky part of using SVM RBF is a good first optionโ€ฆ Depends on your dataโ€”try several. โ€ข Kernels have even been developed for nonnumeric data like sequences, structures, and trees/graphs. May help to use a combination of several kernels. Donโ€™t touch your evaluation data while youโ€™re trying out different kernels and parameters. โ€“ Use cross-validation for this if youโ€™re short on data
Deriving Knowledge from Data at Scale Complexity of the optimization problem remains only dependent on the dimensionality of the input space and not of the feature space!
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale โ€ข SVM 1 learns โ€œOutput==1โ€ vs โ€œOutput != 1โ€ โ€ข SVM 2 learns โ€œOutput==2โ€ vs โ€œOutput != 2โ€ โ€ฆ. โ€ข SVM N learns โ€œOutput==Nโ€ vs โ€œOutput != Nโ€
Deriving Knowledge from Data at Scale2016/4/12 51
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale โ€ข weka.classifiers.functions.SMO โ€ข weka.classifiers.functions.libSVM โ€ข weka.classifiers.functions.SMOreg
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale IF pclass=โ€˜1โ€™ THEN survive=yes ELSE IF pclass=โ€˜2โ€™ THEN survive=yes ELSE IF pclass=โ€˜3โ€™ THEN survive=no confusion matrix no yes <-- classified as 372 119 | no 177 223 | yes (372 + 223) / (372+119+223+177) = 67% correct (and 33% incorrect) a little worse
Deriving Knowledge from Data at Scale Support Vector Machine Model, Titanic Data, Linear Kernel
Deriving Knowledge from Data at Scale Support Vector Machine Model, RBF Kernel Titanic Data
Deriving Knowledge from Data at Scale Support Vector Machine Model, RBF Kernel Titanic Data overfitting?
Deriving Knowledge from Data at Scale Bill Howe, UW 63 Support Vector Machine Model, RBF Kernel Titanic Data A gamma, parameter that controls/balances model complexity against accuracy
Deriving Knowledge from Data at Scale How They Won It! Lessons from data mining the past Kaggle contestsโ€ฆ
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale 10 Minute Breakโ€ฆ
Deriving Knowledge from Data at Scale Data Wrangling
Deriving Knowledge from Data at Scale gold standard data sets
Deriving Knowledge from Data at Scale one dominant attribute even distribution
Deriving Knowledge from Data at Scale โ€ข Rule of thumb: 5,000 or more desired โ€ข Rule of thumb: for each attribute, 10 or more instances โ€ข Rule of thumb: >100 for each class
Deriving Knowledge from Data at Scale Data cleaning Data integration Data transformation Data reduction Data discretization
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale 1. Missing values 2. Outliers 3. Coding 4. Constraints
Deriving Knowledge from Data at Scale ReplaceMissingValues โ€ข RemoveMisclassified โ€ข MergeTwoValues
Deriving Knowledge from Data at Scale Missing values โ€“ UCI machine learning repository, 31 of 68 data sets reported to have missing values. โ€œMissingโ€ can mean many thingsโ€ฆ MAR: "Missing at Random": โ€“ usually best case โ€“ usually not true Non-randomly missing Presumed normal, so not measured Causally missing โ€“ attribute value is missing because of other attribute values (or because of the outcome value!)
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Representation mattersโ€ฆ
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Bank Data A Bank Data B Q Q
Deriving Knowledge from Data at Scale Simple transformations can often have a large impact in performance Example transformations (not all for performance improvement): โ€ข Difference of two date attributes, distance between coordinate,โ€ฆ โ€ข Ratio of two numeric (ratioscale) attributes, average for smoothing,โ€ฆ. โ€ข Concatenating the values of nominal attributes โ€ข Encoding (probabilistic) cluster membership โ€ข Adding noise to data (for robustness tests) โ€ข Removing data randomly or selectively โ€ข Obfuscating the data (for anonymity) Intuition: add features that increase class discrimination (E, IG)โ€ฆ Data Transformation
Deriving Knowledge from Data at Scale โ€ข Combine attributes โ€ข Normalizing data โ€ข Simplifying data
Deriving Knowledge from Data at Scale Change of scale 1 1 1 1 1
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Aggregated data tends to have less variability and potentially more information for better predictionsโ€ฆ
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale fills in the missing values for an instance with the expected values
Deriving Knowledge from Data at Scale Specify the new orderingโ€ฆ 2-last, 1 1-5, 7-last, 6
Deriving Knowledge from Data at Scale instance Resample, a random subsample with or without replacement; To replace or notโ€ฆ Same random seed, will result in same (repeatable) sample. Sample size, as percentage of original data set size.
Deriving Knowledge from Data at Scale Suggestions for you to tryโ€ฆ
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Curse of Dimensionality exponentially In many cases the information that is lost by discarding variables is made up for by a more accurate mapping/sampling in the lower- dimensional space !
Deriving Knowledge from Data at Scale โ€ข Sampling
Deriving Knowledge from Data at Scale work almost as well as using the entire data set the same property (of interest) as the original set of data
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale โ€ข Principle Component Analysis
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale find k โ‰ค n new features (principal components) that can best represent data ๏ƒ  Works for numeric data onlyโ€ฆ
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Attribute Selection (feature selection)
Deriving Knowledge from Data at Scale What is Feature selection ? DIMENSIONALITY REDUCTION
Deriving Knowledge from Data at Scale Problem: Where to focus attention?
Deriving Knowledge from Data at Scale Feature Selection, starts with youโ€ฆ smallest subset of attributes
Deriving Knowledge from Data at Scale What is Evaluated? Attributes Subsets of Attributes Evaluation Method Independent Filters Filters Learning Algorithm Wrappers
Deriving Knowledge from Data at Scale What is Evaluated? Attributes Subsets of Attributes Evaluation Method Independent Filters Filters Learning Algorithm Wrappers
Deriving Knowledge from Data at Scale list of attributes evaluated individually select subset
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale A correlation coefficient shows the degree of linear dependence of x and y. In other words, the coefficient shows how close two variables lie along a line. If the coefficient is equal to 1 or -1, all the points lie along a line. If the correlation coefficient is equal to zero, there is no linear relation between x and y. However, this does not necessarily mean that there is no relation between the two variables. There could e.g. be a non-linear relation.
Deriving Knowledge from Data at Scale Tab for selecting attributes in a data setโ€ฆ
Deriving Knowledge from Data at Scale Interface for classes that evaluate attributesโ€ฆ Interface for ranking or searching for a subset of attributesโ€ฆ
Deriving Knowledge from Data at Scale Select CorrelationAttributeEval for Pearson Correlationโ€ฆ False, doesnโ€™t return R score True, returns R scores;
Deriving Knowledge from Data at Scale Ranks attributes by their individual evaluations, used in conjunction with GainRatio, Entropy, Pearson, etcโ€ฆ Number of attributes to return, -1 returns all ranked attributes; Attributes to ignore (skip) in the evaluation forma: [1, 3-5, 10]; Cutoff at which attributes can be discarded, -1 no cutoff;
Deriving Knowledge from Data at Scale Predicting Self-Reported Health Status The Data Set, NHANES_data.csv (National Health and Nutrition Examination Survey) How would you say your health in general is? Excellent predictor of mortality, health care utilization & disability How I processed itโ€ฆ โ€ข 4000 variables; โ€ข Attributes with > 30% missing values removed (dropped column); โ€ข 105 variables remaining; โ€ข Chi-square test, variable and target, remove variables with P value < .20; โ€ข Impute all missing values using expectation minimization; โ€ข 85 variables remaining; Pearson Correlation Exerciseโ€ฆ
Deriving Knowledge from Data at Scale NHANES_data.csv
Deriving Knowledge from Data at Scale NHANES_data.csv โ€ข Convert the last column from numeric to nominal โ€ข Find the top 15 features using Pearson Correlation
Deriving Knowledge from Data at Scale โ€ข OneRAttributeEval โ€ข GainRatioAttributeEval โ€ข InfoGainAttributeEval โ€ข ChiSquaredAttributeEval โ€ข ReliefFAttributeEval
Deriving Knowledge from Data at Scale โ€ข Right Click on the new line in the Result list; โ€ข From the pop-up menu, select the item Save reduced dataโ€ฆ
Deriving Knowledge from Data at Scale โ€ข Right Click on the new line in the Result list; โ€ข From the pop-up menu, select the item Save reduced dataโ€ฆ โ€ข Save the dataset with 15 selected attributes to file NHanesPearson.arff
Deriving Knowledge from Data at Scale โ€ข Right Click on the new line in the Result list; โ€ข From the pop-up menu, select the item Save reduced dataโ€ฆ โ€ข Save the dataset with 15 selected attributes to file NHanesPearson.arff โ€ข Switch to the Preprocess mode in Explorer โ€ข Click on Open fileโ€ฆ and open the file NHanesPearson.arff โ€ข Switch to the Classify submode โ€ข Click on Choose, select classifier and use this feature set and data to build a predictive model;
Deriving Knowledge from Data at Scale โ€ข Anything below 0.3 isnโ€™t highly correlated with the targetโ€ฆ
Deriving Knowledge from Data at Scale What is Evaluated? Attributes Subsets of Attributes Evaluation Method Independent Filters Filters Learning Algorithm Wrappers
Deriving Knowledge from Data at Scale given the already picked features independent
Deriving Knowledge from Data at Scale all makes the least contribution independent
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Interface for classes that evaluate attributesโ€ฆ Interface for ranking or searching for a subset of attributesโ€ฆ
Deriving Knowledge from Data at Scale
Deriving Knowledge from Data at Scale Forward, Backward, Bi-Directional Attributes to โ€œseedโ€ the search, listed individually or by range. Cutoff for backtrackingโ€ฆ
Deriving Knowledge from Data at Scale True: Adds features that are correlated with class and NOT intercorrelated with other features already in selection. False: Eliminates redundant features. Precompute the correlation matrix in advance, useful for fast backtracking, or compute lazily. When given a large number of attributes, compute lazilyโ€ฆ CfsSubsetEval
Deriving Knowledge from Data at Scale NHANES_data.csv โ€ข Convert the last column from numeric to nominal โ€ข Set the search method as Best First, Forward โ€ข Set the attribute evaluator as CfsSubsetEval โ€ข Run across all attributes in data setโ€ฆ
Deriving Knowledge from Data at Scale โ€ข Feature selection can significantly increase the performance of a learning algorithm (both accuracy and computation time) โ€“ but it is not easy! โ€ข Relevance <-> Optimality โ€ข Correlation and Mutual information between single variables and the target are often used as Ranking-Criteria of variables. Important points 1/2
Deriving Knowledge from Data at Scale Important points 2/2 โ€ข One can not automatically discard variables with small scores โ€“ they may still be useful together with other variables. โ€ข Filters โ€“ Wrappers - Embedded Methods โ€ข How to search the space of all feature subsets ? โ€ข How to asses performance of a learner that uses a particular feature subset ?
Deriving Knowledge from Data at Scale not all about accuracy โ€ข Filtering is fast linear intuitive โ€ข Filtering model oblivious may not be optimal โ€ข Wrappers model-aware slow nonintuitive โ€ข PCA and SVD are lossy work on the entire data set start with fast feature filtering first NOT to use any feature selection
Deriving Knowledge from Data at Scale Thatโ€™s all for tonightโ€ฆ

More Related Content

PDF
Barga Data Science lecture 9
byRoger Barga
ย 
PDF
Barga Data Science lecture 10
byRoger Barga
ย 
PDF
Barga Data Science lecture 8
byRoger Barga
ย 
PDF
Barga Data Science lecture 5
byRoger Barga
ย 
PDF
Barga Data Science lecture 4
byRoger Barga
ย 
PDF
Barga Data Science lecture 3
byRoger Barga
ย 
PDF
Barga Data Science lecture 1
byRoger Barga
ย 
PDF
Barga DIDC'14 Invited Talk
byRoger Barga
ย 
Barga Data Science lecture 9
byRoger Barga
ย 
Barga Data Science lecture 10
byRoger Barga
ย 
Barga Data Science lecture 8
byRoger Barga
ย 
Barga Data Science lecture 5
byRoger Barga
ย 
Barga Data Science lecture 4
byRoger Barga
ย 
Barga Data Science lecture 3
byRoger Barga
ย 
Barga Data Science lecture 1
byRoger Barga
ย 
Barga DIDC'14 Invited Talk
byRoger Barga
ย 

What's hot

PDF
Managing machine learning
byDavid Murgatroyd
ย 
PDF
Introduction to machine learning and deep learning
byShishir Choudhary
ย 
PPTX
H2O World - Top 10 Data Science Pitfalls - Mark Landry
bySri Ambati
ย 
PDF
Machine Learning for Dummies
byVenkata Reddy Konasani
ย 
PDF
Module 5: Decision Trees
bySara Hooker
ย 
PDF
Fairly Measuring Fairness In Machine Learning
byHJ van Veen
ย 
PPTX
Machine Learning and Real-World Applications
byMachinePulse
ย 
PDF
H2O World - Top 10 Deep Learning Tips & Tricks - Arno Candel
bySri Ambati
ย 
PDF
Module 4: Model Selection and Evaluation
bySara Hooker
ย 
PDF
Hacking Predictive Modeling - RoadSec 2018
byHJ van Veen
ย 
PDF
Module 1.2 data preparation
bySara Hooker
ย 
PPTX
Ml1 introduction to-supervised_learning_and_k_nearest_neighbors
byankit_ppt
ย 
PDF
Applications in Machine Learning
byJoel Graff
ย 
PDF
Module 7: Unsupervised Learning
bySara Hooker
ย 
PDF
Module 2: Machine Learning Deep Dive
bySara Hooker
ย 
PPTX
Tips and tricks to win kaggle data science competitions
byDarius Baruลกauskas
ย 
PPTX
Introduction to Machine Learning
byLior Rokach
ย 
PDF
Module 3: Linear Regression
bySara Hooker
ย 
PDF
Feature Reduction Techniques
byVishal Patel
ย 
PDF
Kaggle presentation
byHJ van Veen
ย 
Managing machine learning
byDavid Murgatroyd
ย 
Introduction to machine learning and deep learning
byShishir Choudhary
ย 
H2O World - Top 10 Data Science Pitfalls - Mark Landry
bySri Ambati
ย 
Machine Learning for Dummies
byVenkata Reddy Konasani
ย 
Module 5: Decision Trees
bySara Hooker
ย 
Fairly Measuring Fairness In Machine Learning
byHJ van Veen
ย 
Machine Learning and Real-World Applications
byMachinePulse
ย 
H2O World - Top 10 Deep Learning Tips & Tricks - Arno Candel
bySri Ambati
ย 
Module 4: Model Selection and Evaluation
bySara Hooker
ย 
Hacking Predictive Modeling - RoadSec 2018
byHJ van Veen
ย 
Module 1.2 data preparation
bySara Hooker
ย 
Ml1 introduction to-supervised_learning_and_k_nearest_neighbors
byankit_ppt
ย 
Applications in Machine Learning
byJoel Graff
ย 
Module 7: Unsupervised Learning
bySara Hooker
ย 
Module 2: Machine Learning Deep Dive
bySara Hooker
ย 
Tips and tricks to win kaggle data science competitions
byDarius Baruลกauskas
ย 
Introduction to Machine Learning
byLior Rokach
ย 
Module 3: Linear Regression
bySara Hooker
ย 
Feature Reduction Techniques
byVishal Patel
ย 
Kaggle presentation
byHJ van Veen
ย 

Similar to Barga Data Science lecture 7

PDF
Barga Data Science lecture 6
byRoger Barga
ย 
PDF
Barga Data Science lecture 2
byRoger Barga
ย 
PDF
Introduction to Data Science
byNiko Vuokko
ย 
PPT
594503964-Introduction-to-Classification-PPT-Slides-1.ppt
bysnehajuly2004
ย 
PPT
[ppt]
bybutest
ย 
PPT
[ppt]
bybutest
ย 
PPTX
Supervised learning
byAlia Hamwi
ย 
PPT
Introduction
bybutest
ย 
PPT
Statistical Machine________ Learning.ppt
bySandeepGupta229023
ย 
PPT
Machine Learning and Inductive Inference
bybutest
ย 
PPT
Introduction to Machine Learning Aristotelis Tsirigos
bybutest
ย 
PPTX
Think-Aloud Protocols
bybutest
ย 
PDF
know Machine Learning Basic Concepts.pdf
byhemangppatel
ย 
PDF
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
byMostafaHazemMostafaa
ย 
PDF
Machine Learning ebook.pdf
byHODIT12
ย 
PDF
Machine Learning Final Summary Cheat Sheet
byKristiyana2
ย 
PPTX
Lecture 09(introduction to machine learning)
byJeet Das
ย 
PPT
ai4.ppt
byssuser448ad3
ย 
PPT
ai4.ppt
byatul404633
ย 
PDF
Knowledge discovery claudiad amato
bySSSW
ย 
Barga Data Science lecture 6
byRoger Barga
ย 
Barga Data Science lecture 2
byRoger Barga
ย 
Introduction to Data Science
byNiko Vuokko
ย 
594503964-Introduction-to-Classification-PPT-Slides-1.ppt
bysnehajuly2004
ย 
[ppt]
bybutest
ย 
[ppt]
bybutest
ย 
Supervised learning
byAlia Hamwi
ย 
Introduction
bybutest
ย 
Statistical Machine________ Learning.ppt
bySandeepGupta229023
ย 
Machine Learning and Inductive Inference
bybutest
ย 
Introduction to Machine Learning Aristotelis Tsirigos
bybutest
ย 
Think-Aloud Protocols
bybutest
ย 
know Machine Learning Basic Concepts.pdf
byhemangppatel
ย 
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
byMostafaHazemMostafaa
ย 
Machine Learning ebook.pdf
byHODIT12
ย 
Machine Learning Final Summary Cheat Sheet
byKristiyana2
ย 
Lecture 09(introduction to machine learning)
byJeet Das
ย 
ai4.ppt
byssuser448ad3
ย 
ai4.ppt
byatul404633
ย 
Knowledge discovery claudiad amato
bySSSW
ย 

More from Roger Barga

PDF
RS Barga STRATA'18 New York City
byRoger Barga
ย 
PDF
Barga Strata'18 presentation
byRoger Barga
ย 
PDF
Barga ACM DEBS 2013 Keynote
byRoger Barga
ย 
PDF
Data Driven Engineering 2014
byRoger Barga
ย 
PDF
Barga Galvanize Sept 2015
byRoger Barga
ย 
PDF
Barga IC2E & IoTDI'16 Keynote
byRoger Barga
ย 
RS Barga STRATA'18 New York City
byRoger Barga
ย 
Barga Strata'18 presentation
byRoger Barga
ย 
Barga ACM DEBS 2013 Keynote
byRoger Barga
ย 
Data Driven Engineering 2014
byRoger Barga
ย 
Barga Galvanize Sept 2015
byRoger Barga
ย 
Barga IC2E & IoTDI'16 Keynote
byRoger Barga
ย 

Recently uploaded

PPTX
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
ย 
PDF
SF User Group - Account-Contact Relationship Feb 2026
byLukas Lunow
ย 
PPTX
Toxicity symptoms in plants due to higher concentration
byranveer02yadav
ย 
PDF
"Master Excel with CA Suvidha Chaplot: 40 Must-Know Tips for Efficiency"
byCA Suvidha Chaplot
ย 
PDF
"Top Excel Tips and Tricks: Conditional Formatting, Sorting, and More - Learn...
byCA Suvidha Chaplot
ย 
PPTX
Internship ppt about multiple disease prediction
byGOUTHAMTr2
ย 
PDF
์ œ 23ํšŒ ๋ณด์•„์ฆˆ(BOAZ) ๋น…๋ฐ์ดํ„ฐ ์ปจํผ๋Ÿฐ์Šค - [MBOAX] : ABSA๋ฅผ ํ™œ์šฉํ•œ ์†Œ๋น„์ž ๋ฐ˜์‘ ๋ถ„์„ ๊ธฐ๋ฐ˜ ์šด์˜ ํšจ์œจํ™” ๋Œ€์‹œ๋ณด๋“œ ์„ค๊ณ„
byBOAZ Bigdata
ย 
PDF
CodeMate_Autonomous_Engineering_Documents.pdf
byssuserc287c9
ย 
PDF
Social Media in Gurgaon: Strategies, Services & Solutions by Bala Infotech
byBala Infotech
ย 
PPTX
Smart Crop prediction presentation for final year
bySurya386647
ย 
PPTX
maaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
byAshleyJane9
ย 
PPTX
SQL FOR DATA ANALYSTS - MSSQLServer for Analysts
bysirtwumasi77
ย 
PDF
TOS in History.pdf ENGLISH macro.pdf mnbv
bykristinegorre7
ย 
PDF
"Essential Excel Tips for Beginners & Professionals โ€“ 50+ Techniques to Maste...
byCA Suvidha Chaplot
ย 
PDF
Top 10 Excel Tips for Efficiency | Learn Advanced Excel Techniques CA SUVIDHA...
byCA Suvidha Chaplot
ย 
PPTX
Hacking-Uncovered-How-People-Get-Hacked-and-How-to-Stay-Safe[1].pptx
bylegendgaminghari
ย 
PDF
Baltic Startup Funding Report 2025 by FIRSTPICK and Practica Capital
byFIRSTPICK VC
ย 
PPTX
Autism types, forms and everything we need to know
byfuzertamas
ย 
PPT
Data Warehousing and Data Mining โ€“ Concepts, Architecture and Applications
bystark317700
ย 
PPTX
Microsoft-Dynamics-365-Business-Central (1).pptx
byUlirRohwana
ย 
Query Parsing: The SEO Gatekeeper That Decides Rankings
bySemantic SEO BD
ย 
SF User Group - Account-Contact Relationship Feb 2026
byLukas Lunow
ย 
Toxicity symptoms in plants due to higher concentration
byranveer02yadav
ย 
"Master Excel with CA Suvidha Chaplot: 40 Must-Know Tips for Efficiency"
byCA Suvidha Chaplot
ย 
"Top Excel Tips and Tricks: Conditional Formatting, Sorting, and More - Learn...
byCA Suvidha Chaplot
ย 
Internship ppt about multiple disease prediction
byGOUTHAMTr2
ย 
์ œ 23ํšŒ ๋ณด์•„์ฆˆ(BOAZ) ๋น…๋ฐ์ดํ„ฐ ์ปจํผ๋Ÿฐ์Šค - [MBOAX] : ABSA๋ฅผ ํ™œ์šฉํ•œ ์†Œ๋น„์ž ๋ฐ˜์‘ ๋ถ„์„ ๊ธฐ๋ฐ˜ ์šด์˜ ํšจ์œจํ™” ๋Œ€์‹œ๋ณด๋“œ ์„ค๊ณ„
byBOAZ Bigdata
ย 
CodeMate_Autonomous_Engineering_Documents.pdf
byssuserc287c9
ย 
Social Media in Gurgaon: Strategies, Services & Solutions by Bala Infotech
byBala Infotech
ย 
Smart Crop prediction presentation for final year
bySurya386647
ย 
maaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
byAshleyJane9
ย 
SQL FOR DATA ANALYSTS - MSSQLServer for Analysts
bysirtwumasi77
ย 
TOS in History.pdf ENGLISH macro.pdf mnbv
bykristinegorre7
ย 
"Essential Excel Tips for Beginners & Professionals โ€“ 50+ Techniques to Maste...
byCA Suvidha Chaplot
ย 
Top 10 Excel Tips for Efficiency | Learn Advanced Excel Techniques CA SUVIDHA...
byCA Suvidha Chaplot
ย 
Hacking-Uncovered-How-People-Get-Hacked-and-How-to-Stay-Safe[1].pptx
bylegendgaminghari
ย 
Baltic Startup Funding Report 2025 by FIRSTPICK and Practica Capital
byFIRSTPICK VC
ย 
Autism types, forms and everything we need to know
byfuzertamas
ย 
Data Warehousing and Data Mining โ€“ Concepts, Architecture and Applications
bystark317700
ย 
Microsoft-Dynamics-365-Business-Central (1).pptx
byUlirRohwana
ย 

Barga Data Science lecture 7

  • 1.
  • 2.
    Deriving Knowledge fromData at Scale Good data preparation is key to producing valid and reliable modelsโ€ฆ
  • 3.
    Deriving Knowledge fromData at Scale Lecture 7 Agenda
  • 4.
  • 5.
  • 6.
  • 7.
    Deriving Knowledge fromData at Scale Heads Up, Lecture 8 Agenda
  • 8.
    Deriving Knowledge fromData at Scale Course Project
  • 9.
  • 10.
    Deriving Knowledge fromData at Scale https://www.kaggle.com/c/springleaf-marketing-response not Determine whether to send a direct mail piece to a customer
  • 11.
    Deriving Knowledge fromData at Scale The Data
  • 12.
    Deriving Knowledge fromData at Scale The Rules
  • 13.
  • 14.
    Deriving Knowledge fromData at Scale what is the data telling you
  • 15.
  • 16.
    Deriving Knowledge fromData at Scale The Tragedy of the Titanic
  • 17.
  • 18.
  • 19.
  • 20.
    Deriving Knowledge fromData at Scale Does the new data point x* exactly match a previous point xi? If so, assign it to the same class as xi Otherwise, just guess. This is the โ€œroteโ€ classifier
  • 21.
    Deriving Knowledge fromData at Scale Does the new data point x* match a set pf previous points xi on some specific attribute? If so, take a vote to determine class. Example: If most females survived, then assume every female survives But there are lots of possible rules like this. And an attribute can have more than two values. If most people under 4 years old survive, then assume everyone under 4 survives If most people with 1 sibling survive, then assume everyone with 1 sibling survives How do we choose?
  • 22.
    Deriving Knowledge fromData at Scale IF sex=โ€˜femaleโ€™ THEN survive=yes ELSE IF sex=โ€˜maleโ€™ THEN survive = no confusion matrix no yes <-- classified as 468 109 | no 81 233 | yes (468 + 233) / (468+109+81+233) = 79% correct (and 21% incorrect) Not bad!
  • 23.
  • 24.
    Deriving Knowledge fromData at Scale IF pclass=โ€˜1โ€™ THEN survive=yes ELSE IF pclass=โ€˜2โ€™ THEN survive=yes ELSE IF pclass=โ€˜3โ€™ THEN survive=no confusion matrix no yes <-- classified as 372 119 | no 177 223 | yes (372 + 223) / (372+119+223+177) = 67% correct (and 33% incorrect) a little worse
  • 25.
    Deriving Knowledge fromData at Scale Support Vector Machines
  • 26.
  • 27.
  • 28.
    Deriving Knowledge fromData at Scale fx yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data? Estimation: w: weight vector x: data vector
  • 29.
    Deriving Knowledge fromData at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?
  • 30.
    Deriving Knowledge fromData at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?
  • 31.
    Deriving Knowledge fromData at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) How would you classify this data?
  • 32.
    Deriving Knowledge fromData at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Any of these would be fine.. ..but which is best?
  • 33.
    Deriving Knowledge fromData at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Define the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.
  • 34.
    Deriving Knowledge fromData at Scale O X O O O X X X SUPPORT VECTORS
  • 35.
    Deriving Knowledge fromData at Scale O X X O O O X X X SUPPORT VECTORS
  • 36.
    Deriving Knowledge fromData at Scale fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM) Linear SVM
  • 37.
    Deriving Knowledge fromData at Scale2016/4/12 37 fx a yest denotes +1 denotes -1 f(x,w,b) = sign(w. x + b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM)Support Vectors are those datapoints that the margin pushes up against Linear SVM
  • 38.
    Deriving Knowledge fromData at Scale denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) Support Vectors are those datapoints that the margin pushes up against 1. Intuitively this feels safest. 2. If weโ€™ve made a small error in the location of the boundary this gives us least chance of causing a misclassification. 3. LOOCV (leave one out cross validation) is easy since the model is immune to removal of any nonsupport-vector data points. 5. Empirically it works very well.
  • 39.
    Deriving Knowledge fromData at Scale Class 1 Class 2 m
  • 40.
    Deriving Knowledge fromData at Scale This is going to be a problem!
  • 41.
  • 42.
    Deriving Knowledge fromData at Scale What if the data looks like that below?... kernel function
  • 43.
    Deriving Knowledge fromData at Scale What if the data looks like that below?...
  • 44.
    Deriving Knowledge fromData at Scale Everything you need to know in one slideโ€ฆ
  • 45.
  • 46.
  • 47.
    Deriving Knowledge fromData at Scale Probably the most tricky part of using SVM RBF is a good first optionโ€ฆ Depends on your dataโ€”try several. โ€ข Kernels have even been developed for nonnumeric data like sequences, structures, and trees/graphs. May help to use a combination of several kernels. Donโ€™t touch your evaluation data while youโ€™re trying out different kernels and parameters. โ€“ Use cross-validation for this if youโ€™re short on data
  • 48.
    Deriving Knowledge fromData at Scale Complexity of the optimization problem remains only dependent on the dimensionality of the input space and not of the feature space!
  • 49.
  • 50.
    Deriving Knowledge fromData at Scale โ€ข SVM 1 learns โ€œOutput==1โ€ vs โ€œOutput != 1โ€ โ€ข SVM 2 learns โ€œOutput==2โ€ vs โ€œOutput != 2โ€ โ€ฆ. โ€ข SVM N learns โ€œOutput==Nโ€ vs โ€œOutput != Nโ€
  • 51.
    Deriving Knowledge fromData at Scale2016/4/12 51
  • 52.
  • 53.
  • 54.
  • 55.
    Deriving Knowledge fromData at Scale โ€ข weka.classifiers.functions.SMO โ€ข weka.classifiers.functions.libSVM โ€ข weka.classifiers.functions.SMOreg
  • 56.
  • 57.
  • 58.
  • 59.
    Deriving Knowledge fromData at Scale IF pclass=โ€˜1โ€™ THEN survive=yes ELSE IF pclass=โ€˜2โ€™ THEN survive=yes ELSE IF pclass=โ€˜3โ€™ THEN survive=no confusion matrix no yes <-- classified as 372 119 | no 177 223 | yes (372 + 223) / (372+119+223+177) = 67% correct (and 33% incorrect) a little worse
  • 60.
    Deriving Knowledge fromData at Scale Support Vector Machine Model, Titanic Data, Linear Kernel
  • 61.
    Deriving Knowledge fromData at Scale Support Vector Machine Model, RBF Kernel Titanic Data
  • 62.
    Deriving Knowledge fromData at Scale Support Vector Machine Model, RBF Kernel Titanic Data overfitting?
  • 63.
    Deriving Knowledge fromData at Scale Bill Howe, UW 63 Support Vector Machine Model, RBF Kernel Titanic Data A gamma, parameter that controls/balances model complexity against accuracy
  • 64.
    Deriving Knowledge fromData at Scale How They Won It! Lessons from data mining the past Kaggle contestsโ€ฆ
  • 65.
  • 66.
    Deriving Knowledge fromData at Scale 10 Minute Breakโ€ฆ
  • 67.
    Deriving Knowledge fromData at Scale Data Wrangling
  • 68.
    Deriving Knowledge fromData at Scale gold standard data sets
  • 69.
    Deriving Knowledge fromData at Scale one dominant attribute even distribution
  • 70.
    Deriving Knowledge fromData at Scale โ€ข Rule of thumb: 5,000 or more desired โ€ข Rule of thumb: for each attribute, 10 or more instances โ€ข Rule of thumb: >100 for each class
  • 71.
    Deriving Knowledge fromData at Scale Data cleaning Data integration Data transformation Data reduction Data discretization
  • 72.
  • 73.
    Deriving Knowledge fromData at Scale 1. Missing values 2. Outliers 3. Coding 4. Constraints
  • 74.
    Deriving Knowledge fromData at Scale ReplaceMissingValues โ€ข RemoveMisclassified โ€ข MergeTwoValues
  • 75.
    Deriving Knowledge fromData at Scale Missing values โ€“ UCI machine learning repository, 31 of 68 data sets reported to have missing values. โ€œMissingโ€ can mean many thingsโ€ฆ MAR: "Missing at Random": โ€“ usually best case โ€“ usually not true Non-randomly missing Presumed normal, so not measured Causally missing โ€“ attribute value is missing because of other attribute values (or because of the outcome value!)
  • 76.
  • 77.
    Deriving Knowledge fromData at Scale Representation mattersโ€ฆ
  • 78.
  • 79.
  • 80.
    Deriving Knowledge fromData at Scale Bank Data A Bank Data B Q Q
  • 81.
    Deriving Knowledge fromData at Scale Simple transformations can often have a large impact in performance Example transformations (not all for performance improvement): โ€ข Difference of two date attributes, distance between coordinate,โ€ฆ โ€ข Ratio of two numeric (ratioscale) attributes, average for smoothing,โ€ฆ. โ€ข Concatenating the values of nominal attributes โ€ข Encoding (probabilistic) cluster membership โ€ข Adding noise to data (for robustness tests) โ€ข Removing data randomly or selectively โ€ข Obfuscating the data (for anonymity) Intuition: add features that increase class discrimination (E, IG)โ€ฆ Data Transformation
  • 82.
    Deriving Knowledge fromData at Scale โ€ข Combine attributes โ€ข Normalizing data โ€ข Simplifying data
  • 83.
    Deriving Knowledge fromData at Scale Change of scale 1 1 1 1 1
  • 84.
  • 85.
  • 86.
    Deriving Knowledge fromData at Scale Aggregated data tends to have less variability and potentially more information for better predictionsโ€ฆ
  • 87.
  • 88.
  • 89.
  • 90.
  • 91.
  • 92.
  • 93.
    Deriving Knowledge fromData at Scale fills in the missing values for an instance with the expected values
  • 94.
    Deriving Knowledge fromData at Scale Specify the new orderingโ€ฆ 2-last, 1 1-5, 7-last, 6
  • 95.
    Deriving Knowledge fromData at Scale instance Resample, a random subsample with or without replacement; To replace or notโ€ฆ Same random seed, will result in same (repeatable) sample. Sample size, as percentage of original data set size.
  • 96.
    Deriving Knowledge fromData at Scale Suggestions for you to tryโ€ฆ
  • 97.
  • 98.
    Deriving Knowledge fromData at Scale Curse of Dimensionality exponentially In many cases the information that is lost by discarding variables is made up for by a more accurate mapping/sampling in the lower- dimensional space !
  • 99.
    Deriving Knowledge fromData at Scale โ€ข Sampling
  • 100.
    Deriving Knowledge fromData at Scale work almost as well as using the entire data set the same property (of interest) as the original set of data
  • 101.
  • 102.
  • 103.
    Deriving Knowledge fromData at Scale โ€ข Principle Component Analysis
  • 104.
  • 105.
    Deriving Knowledge fromData at Scale find k โ‰ค n new features (principal components) that can best represent data ๏ƒ  Works for numeric data onlyโ€ฆ
  • 106.
  • 107.
    Deriving Knowledge fromData at Scale Attribute Selection (feature selection)
  • 108.
    Deriving Knowledge fromData at Scale What is Feature selection ? DIMENSIONALITY REDUCTION
  • 109.
    Deriving Knowledge fromData at Scale Problem: Where to focus attention?
  • 110.
    Deriving Knowledge fromData at Scale Feature Selection, starts with youโ€ฆ smallest subset of attributes
  • 111.
    Deriving Knowledge fromData at Scale What is Evaluated? Attributes Subsets of Attributes Evaluation Method Independent Filters Filters Learning Algorithm Wrappers
  • 112.
    Deriving Knowledge fromData at Scale What is Evaluated? Attributes Subsets of Attributes Evaluation Method Independent Filters Filters Learning Algorithm Wrappers
  • 113.
    Deriving Knowledge fromData at Scale list of attributes evaluated individually select subset
  • 114.
  • 115.
    Deriving Knowledge fromData at Scale A correlation coefficient shows the degree of linear dependence of x and y. In other words, the coefficient shows how close two variables lie along a line. If the coefficient is equal to 1 or -1, all the points lie along a line. If the correlation coefficient is equal to zero, there is no linear relation between x and y. However, this does not necessarily mean that there is no relation between the two variables. There could e.g. be a non-linear relation.
  • 116.
    Deriving Knowledge fromData at Scale Tab for selecting attributes in a data setโ€ฆ
  • 117.
    Deriving Knowledge fromData at Scale Interface for classes that evaluate attributesโ€ฆ Interface for ranking or searching for a subset of attributesโ€ฆ
  • 118.
    Deriving Knowledge fromData at Scale Select CorrelationAttributeEval for Pearson Correlationโ€ฆ False, doesnโ€™t return R score True, returns R scores;
  • 119.
    Deriving Knowledge fromData at Scale Ranks attributes by their individual evaluations, used in conjunction with GainRatio, Entropy, Pearson, etcโ€ฆ Number of attributes to return, -1 returns all ranked attributes; Attributes to ignore (skip) in the evaluation forma: [1, 3-5, 10]; Cutoff at which attributes can be discarded, -1 no cutoff;
  • 120.
    Deriving Knowledge fromData at Scale Predicting Self-Reported Health Status The Data Set, NHANES_data.csv (National Health and Nutrition Examination Survey) How would you say your health in general is? Excellent predictor of mortality, health care utilization & disability How I processed itโ€ฆ โ€ข 4000 variables; โ€ข Attributes with > 30% missing values removed (dropped column); โ€ข 105 variables remaining; โ€ข Chi-square test, variable and target, remove variables with P value < .20; โ€ข Impute all missing values using expectation minimization; โ€ข 85 variables remaining; Pearson Correlation Exerciseโ€ฆ
  • 121.
    Deriving Knowledge fromData at Scale NHANES_data.csv
  • 122.
    Deriving Knowledge fromData at Scale NHANES_data.csv โ€ข Convert the last column from numeric to nominal โ€ข Find the top 15 features using Pearson Correlation
  • 123.
    Deriving Knowledge fromData at Scale โ€ข OneRAttributeEval โ€ข GainRatioAttributeEval โ€ข InfoGainAttributeEval โ€ข ChiSquaredAttributeEval โ€ข ReliefFAttributeEval
  • 124.
    Deriving Knowledge fromData at Scale โ€ข Right Click on the new line in the Result list; โ€ข From the pop-up menu, select the item Save reduced dataโ€ฆ
  • 125.
    Deriving Knowledge fromData at Scale โ€ข Right Click on the new line in the Result list; โ€ข From the pop-up menu, select the item Save reduced dataโ€ฆ โ€ข Save the dataset with 15 selected attributes to file NHanesPearson.arff
  • 126.
    Deriving Knowledge fromData at Scale โ€ข Right Click on the new line in the Result list; โ€ข From the pop-up menu, select the item Save reduced dataโ€ฆ โ€ข Save the dataset with 15 selected attributes to file NHanesPearson.arff โ€ข Switch to the Preprocess mode in Explorer โ€ข Click on Open fileโ€ฆ and open the file NHanesPearson.arff โ€ข Switch to the Classify submode โ€ข Click on Choose, select classifier and use this feature set and data to build a predictive model;
  • 127.
    Deriving Knowledge fromData at Scale โ€ข Anything below 0.3 isnโ€™t highly correlated with the targetโ€ฆ
  • 128.
    Deriving Knowledge fromData at Scale What is Evaluated? Attributes Subsets of Attributes Evaluation Method Independent Filters Filters Learning Algorithm Wrappers
  • 129.
    Deriving Knowledge fromData at Scale given the already picked features independent
  • 130.
    Deriving Knowledge fromData at Scale all makes the least contribution independent
  • 131.
  • 132.
  • 133.
  • 134.
  • 135.
    Deriving Knowledge fromData at Scale Interface for classes that evaluate attributesโ€ฆ Interface for ranking or searching for a subset of attributesโ€ฆ
  • 136.
  • 137.
    Deriving Knowledge fromData at Scale Forward, Backward, Bi-Directional Attributes to โ€œseedโ€ the search, listed individually or by range. Cutoff for backtrackingโ€ฆ
  • 138.
    Deriving Knowledge fromData at Scale True: Adds features that are correlated with class and NOT intercorrelated with other features already in selection. False: Eliminates redundant features. Precompute the correlation matrix in advance, useful for fast backtracking, or compute lazily. When given a large number of attributes, compute lazilyโ€ฆ CfsSubsetEval
  • 139.
    Deriving Knowledge fromData at Scale NHANES_data.csv โ€ข Convert the last column from numeric to nominal โ€ข Set the search method as Best First, Forward โ€ข Set the attribute evaluator as CfsSubsetEval โ€ข Run across all attributes in data setโ€ฆ
  • 140.
    Deriving Knowledge fromData at Scale โ€ข Feature selection can significantly increase the performance of a learning algorithm (both accuracy and computation time) โ€“ but it is not easy! โ€ข Relevance <-> Optimality โ€ข Correlation and Mutual information between single variables and the target are often used as Ranking-Criteria of variables. Important points 1/2
  • 141.
    Deriving Knowledge fromData at Scale Important points 2/2 โ€ข One can not automatically discard variables with small scores โ€“ they may still be useful together with other variables. โ€ข Filters โ€“ Wrappers - Embedded Methods โ€ข How to search the space of all feature subsets ? โ€ข How to asses performance of a learner that uses a particular feature subset ?
  • 142.
    Deriving Knowledge fromData at Scale not all about accuracy โ€ข Filtering is fast linear intuitive โ€ข Filtering model oblivious may not be optimal โ€ข Wrappers model-aware slow nonintuitive โ€ข PCA and SVD are lossy work on the entire data set start with fast feature filtering first NOT to use any feature selection
  • 143.
    Deriving Knowledge fromData at Scale Thatโ€™s all for tonightโ€ฆ