This document summarizes a study that compares different feature selection methods for text categorization. The study aims to reduce the number of dimensions to address issues with high dimensionality. It evaluates several statistical classification methods and feature selection techniques, including document frequency thresholding, information gain, mutual information, CHI statistic, and term strength. The experiments apply k-nearest neighbors and linear least squares fitting classifiers to Reuters and OHSUMED corpora. The results show the best performance with a vocabulary size of around 2,000 terms and that information gain and CHI are the most aggressive at term removal.

1. A Comparative Study on Featuree
Selection in Text Categorization
Presented by Hector Franco
TCD

2. objective
• Reduce the number of dimensions. Some
methods have problems with too high
dimension.

3. Statistical classification methods.
1. Regression models
2. Knn
3. Bayes
4. Decision treees
5. Neural netwoks
6. Symbolic rule learning
7. Inductive learning algorithms

4. Features:
• DF Document frequency thresholding
• IG Information Gain
• MI Mutual information
• CHI statistic
• TS Term strength

5. DF Document frequency thresholding
• Number of documents in which term occurs.
• It remove rare terms.

6. Information gain
• Of the term t:
• Time: O(N) space O(VN)
• N=Documents, V=vocabulary

7. Mutual information
• If t and c indpendent -> value 0.
O(VN)

8. Statistic (CHI)
• Measure of the lack of independence between t
and c,
• A t and c occurs, B t and not c
• C not t and c , D not t and not c
• N total number of documents
It t and c independent value =0.

9. Statistic (CHI)
O(VN)

10. Ts term strength
• Based on document clustering
• How common is a term is likely to appear in
closely related documents.
• O(N^2)

11. EXPERIMENTS
• Classifiers
– kNN
– LLSF
• Corporas:
– Reuters-22173
– OHSUMED
• Use of SMART system for unified
preprocessing.

12. Reduction on number of words
Have the best performance at 2000
vocabulary size
Best ig (more reduction)and chi

13. Most
aggressive in
term removal

14. Creative commons license
You are free:
•to copy, distribute, display, and perform the work
•to make derivative works
Under the following conditions:
•Attribution. You must give the original author credit.
What does quot;Attribute this workquot; mean?
The page you came from contained embedded licensing metadata, including how the creator wishes to be
attributed for re-use. You can use the HTML here to cite the work. Doing so will also include metadata on
your page so that others can find the original work as well.
•Non-Commercial. You may not use this work for commercial purposes.
•For any reuse or distribution, you must make clear to others the licence terms of this work.
•Any of these conditions can be waived if you get permission from the copyright holder.
•Nothing in this license impairs or restricts the author's moral rights.