it is well suited for multi-modal classes as its classification decision is based on a small neighborhood of similar objects (i.e., the major class).
So, even if the target class is multi-modal (i.e., consists of objects whose independent variables have different characteristics for different subsets), it can still lead to good accuracy.
A major drawback of the similarity measure used in KNN is that it uses all features equally in computing similarities. This can lead to poor similarity measures and classification errors, when only a small subset of the features is useful for classification
8.
Decision trees Decision trees are popular for pattern recognition because the models they produce are easier to understand. Root node A A B B B B
Linear decision trees are similar to binary decision trees, except that the inequality computed at each node takes on an arbitrary linear from that may depend on multiple variables.
a naive Bayes classifier assumes that the presence (or absence) of a particular feature of a class is unrelated to the presence (or absence) of any other feature.
Example:
a fruit may be considered to be an apple if it is red, round, and about 4" in diameter. Even though these features depend on the existence of the other features, a naive Bayes classifier considers all of these properties to independently contribute to the probability that this fruit is an apple.
All model parameters can be approximated with relative frequencies from the training set.
If a given class and feature value never occur together in the training set then the frequency-based probability estimate will be zero.
This is problematic since it will wipe out all information in the other probabilities when they are multiplied. It is therefore often desirable to incorporate a small-sample correction in all probability estimates such that no probability is ever set to be exactly zero.
13.
Constructing a classifier from the probability model
The naive Bayes classifier combines this model with a decision rule. One common rule is to pick the hypothesis that is most probable.
This is known as the maximum a posteriori or MAP decision rule. The corresponding classifier is the function classify defined as follows:
Given n test documents and m classes in consideration, a classifier makes n m binary decisions. A two-by-two contingency table can be computed for each class.
Be the first to comment