Algorithm

Linguistics

Philosophy

Statistics

Support vector machine

Binary classification

Data mining

Computer science

Probability distribution

Kullback–Leibler divergence

Divergence (linguistics)

Pattern recognition (psychology)

Mathematics

Statistical hypothesis testing

Nonparametric statistics

Feature selection

Feature (linguistics)

Artificial intelligence

Parametric statistics

Measure (data warehouse)

Binary number

Machine learning

Arithmetic

IEEE Transactions on Signal Processing

Information divergence functions play a critical role in statistics and information theory. In this paper we show that a nonparametric f-divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the same distribution and for the case where there exists some mismatch between training and test distributions. We confirm these theoretical results by designing feature selection algorithms using the criteria from these bounds and by evaluating the algorithms on a series of pathological speech classification tasks.

Empirically Estimable Classification Bounds Based on a Nonparametric Divergence Measure