Learning theory

Function (biology)

Mathematics education

Biology

Statistics

Support vector machine

Theoretical computer science

Statistical theory

Instance-based learning

Computer science

Information theory

Evolutionary biology

Algorithmic learning theory

Probably approximately correct learning

Mathematics

Mathematical proof

Statistical learning theory

Mathematical analysis

Artificial intelligence

Geometry

VC dimension

Computational learning theory

Unsupervised learning

Generalization

Machine learning

IEEE Transactions on Neural Networks

Statistical learning theory was introduced in the late 1960's. Until the 1990's it was a purely theoretical analysis of the problem of function estimation from a given collection of data. In the middle of the 1990's new types of learning algorithms (called support vector machines) based on the developed theory were proposed. This made statistical learning theory not only a tool for the theoretical analysis but also a tool for creating practical algorithms for estimating multidimensional functions. This article presents a very general overview of statistical learning theory including both theoretical and algorithmic aspects of the theory. The goal of this overview is to demonstrate how the abstract learning theory established conditions for generalization which are more general than those discussed in classical statistical paradigms and how the understanding of these conditions inspired new algorithmic approaches to function estimation problems.

An overview of statistical learning theory