Algorithm

Discrete mathematics

Support vector machine

Scalability

Computer science

Mathematical optimization

Nyström method

Kernel (algebra)

Database

Mathematics

Kernel method

Leverage (statistics)

Mathematical analysis

Artificial intelligence

Integral equation

arXiv (Cornell University)

We give the first algorithm for kernel Nystrom approximation that runs in linear time in the number of training points and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions. The algorithm projects the kernel onto a set of s landmark points sampled by their ridge leverage scores, requiring just O(ns) kernel evaluations and O(ns^2) additional runtime. While leverage score sampling has long been known to give strong theoretical guarantees for Nystrom approximation, by employing a fast recursive sampling scheme, our algorithm is the first to make the approach scalable. Empirically we show that it finds more accurate kernel approximations in less time than popular techniques such as classic Nystrom approximation and the random Fourier features method.

Recursive Sampling for the Nyström Method