Pure mathematics

Mathematics

Evolutionary biology

Biology

Sobolev space

Algorithm

Approximation error

Mathematical analysis

Function (biology)

Approximation theory

Political science

Approximation algorithm

Paleontology

Artificial intelligence

Norm (philosophy)

Approximations of π

Context (archaeology)

Computer science

Applied mathematics

Artificial neural network

Upper and lower bounds

Function approximation

Analysis and Applications

We analyze to what extent deep Rectified Linear Unit (ReLU) neural networks can efficiently approximate Sobolev regular functions if the approximation error is measured with respect to weaker Sobolev norms. In this context, we first establish upper approximation bounds by ReLU neural networks for Sobolev regular functions by explicitly constructing the approximate ReLU neural networks. Then, we establish lower approximation bounds for the same type of function classes. A trade-off between the regularity used in the approximation norm and the complexity of the neural network can be observed in upper and lower bounds. Our results extend recent advances in the approximation theory of ReLU networks to the regime that is most relevant for applications in the numerical analysis of partial differential equations.

Error bounds for approximations with deep ReLU neural networks in Ws,p norms