Paper Info
Title
Feature Selection Based on Class-Dependent Densities for High-Dimensional Binary Data
Abstract
Data and knowledge management systems employ feature selection algorithms for removing irrelevant, redundant, and noisy information from the data. There are two well-known approaches to feature selection, feature ranking (FR) and feature subset selection (FSS). In this paper, we propose a new FR algorithm, termed as class-dependent density-based feature elimination (CDFE), for binary data sets. Our theoretical analysis shows that CDFE computes the weights, used for feature ranking, more efficiently as compared to the mutual information measure. Effectively, rankings obtained from both the two criteria approximate each other. CDFE uses a filtrapper approach to select a final subset. For data sets having hundreds of thousands of features, feature selection with FR algorithms is simple and computationally efficient but redundant information may not be removed. On the other hand, FSS algorithms analyze the data for redundancies but may become computationally impractical on high-dimensional data sets. We address these problems by combining FR and FSS methods in the form of a two-stage feature selection algorithm. When introduced as a preprocessing step to the FSS algorithms, CDFE not only presents them with a feature subset, good in terms of classification, but also relieves them from heavy computations. Two FSS algorithms are employed in the second stage to test the two-stage feature selection idea. We carry out experiments with two different classifiers (naive Bayes' and kernel ridge regression) on three different real-life data sets (NOVA, HIVA, and GINA) of the ”Agnostic Learning versus Prior Knowledge” challenge. As a stand-alone method, CDFE shows up to about 92 percent reduction in the feature set size. When combined with the FSS algorithms in two-stages, CDFE significantly improves their classification accuracy and exhibits up to 97 percent reduction in the feature set size. We also compared CDFE against the winning entries of the challenge and found that it outperforms the best results on NOVA and HIVA while obtaining a third position in case of GINA.
Year
DOI
Venue
2012
10.1109/TKDE.2010.263
IEEE Trans. Knowl. Data Eng.
Keywords
Field
DocType
feature selection,feature set size,class-dependent density-based feature elimination,feature ranking,feature subset,two-stage feature selection idea,high-dimensional binary data,class-dependent densities,feature selection algorithm,feature subset selection,two-stage feature selection algorithm,fss algorithm,naive bayes classifier,classification,approximation algorithms,knowledge management system,accuracy,markov processes,mutual information,algorithm design and analysis,high dimensional data,redundancy,markov process,frequency selective surface,algorithm design
Data mining,Data set,Feature selection,Computer science,Artificial intelligence,Approximation algorithm,Algorithm design,Pattern recognition,Naive Bayes classifier,Preprocessor,Mutual information,Binary data,Machine learning
Journal
Volume
Issue
ISSN
24
3
1041-4347
Citations 
PageRank 
References 
40
1.37
19
Authors
3
Name
Order
Citations
PageRank
Kashif Javed11108.87
Haroon A. Babri2814.63
Mehreen Saeed3877.32