Classification and prediction models are used to categorize data or predict unknown values. Classification predicts categorical class labels to classify new data based on attributes in a training set, while prediction models continuous values. Common applications include credit approval, marketing, medical diagnosis, and treatment analysis. The classification process involves building a model from a training set and then using the model to classify new data, estimating accuracy on a test set.

1. Data Mining:
Classification

2. Classification and Prediction
• What is classification? What is prediction?
• Issues regarding classification and prediction
• Classification by decision tree induction

3. • Classification:
– predicts categorical class labels
– classifies data (constructs a model) based on the
training set and the values (class labels) in a
classifying attribute and uses it in classifying new data
• Prediction:
– models continuous-valued functions, i.e., predicts
unknown or missing values
• Typical Applications
– credit approval
– target marketing
– medical diagnosis
– treatment effectiveness analysis
Classification vs. Prediction

4. Classification Learning: Definition
l Given a collection of records (training set)
– Each record contains a set of attributes, one of the
attributes is the class
l Find a model for the class attribute as a function
of the values of the other attributes
l Goal: previously unseen records should be
assigned a class as accurately as possible
– Use test set to estimate the accuracy of the model
– Often, the given data set is divided into training and test
sets, with training set used to build the model and test
set used to validate it.

5. Illustrating Classification Learning
Apply
Model
Induction
Deduction
Learn
Model
Model
Tid Attrib1 Attrib2 Attrib3 Class
1 Yes Large 125K No
2 No Medium 100K No
3 No Small 70K No
4 Yes Medium 120K No
5 No Large 95K Yes
6 No Medium 60K No
7 Yes Large 220K No
8 No Small 85K Yes
9 No Medium 75K No
10 No Small 90K Yes
10
Tid Attrib1 Attrib2 Attrib3 Class
11 No Small 55K ?
12 Yes Medium 80K ?
13 Yes Large 110K ?
14 No Small 95K ?
15 No Large 67K ?
10
Test Set
Learning
algorithm
Training Set

6. Examples of Classification Task
l Predicting tumor cells as benign or malignant
l Classifying credit card transactions
as legitimate or fraudulent
l Classifying secondary structures of protein
as alpha-helix, beta-sheet, or random
coil
l Categorizing news stories as finance,
weather, entertainment, sports, etc.

7. Classification - A Two-Step Process
• Model construction: describing a set of predetermined classes
– Building the Classifier or Model
– Each tuple/sample is assumed to belong to a predefined class, as
determined by the class label attribute
– The set of tuples used for model construction: training set
– The model is represented as classification rules, decision trees, or
mathematical formulae
• Model usage: for classifying future or unknown objects
– Using Classifier for Classification
– Estimate accuracy of the model
• The known label of test sample is compared with the classified result
from the model
• Accuracy rate is the percentage of test set samples that are correctly
classified by the model
• Test set is independent of training set, otherwise over-fitting will
occur

8. Classification Process (1): Model Construction
Example: Loan application
The data classification process: Learning: Training data are analyzed by a
classification algorithm. Here, the class label attribute is loan decision, and the
learned model or classifier is represented in the form of classification rules.

9. Classification Process (2): Use the Model
in Prediction
Classification: Test data are used to estimate the accuracy of the classification
rules. If the accuracy is considered acceptable, the rules can be applied to the
classification of new data tuples.

10. Supervised vs. Unsupervised
Learning
• Supervised learning (classification)
– Supervision: The training data (observations,
measurements, etc.) are accompanied by labels
indicating the class of the observations
– New data is classified based on the training set
• Unsupervised learning (clustering)
– The class labels of training data is unknown
– Given a set of measurements, observations, etc. with
the aim of establishing the existence of classes or
clusters in the data

11. Issues (1): Data Preparation
• Data cleaning
– Preprocess data in order to reduce noise and handle
missing values
• Relevance analysis (feature selection)
– Remove the irrelevant or redundant attributes
• Data transformation
– Generalize and/or normalize data

12. Issues (2): Evaluating Classification
Methods
• Predictive accuracy
• Speed and scalability
– time to construct the model
– time to use the model
• Robustness
– handling noise and missing values
• Scalability
– efficiency in disk-resident databases
• Interpretability:
– understanding and insight provded by the model
• Goodness of rules
– decision tree size
– compactness of classification rules

13. The problem
• Given a set of training cases/objects and their attribute
values, try to determine the target attribute value of new
examples.
– Classification
– Prediction
• Use a decision tree to predict categories for new events.
• Use training data to build the decision tree.
New
Events
Decision
Tree
Category
Training
Events and
Categories