Nonlinear programming

Identification (biology)

Physics

Regression analysis

Nonlinear system

Mathematics

Nonlinear regression

Biology

Genetic algorithm

Genetic programming

Operating system

Algorithm

Mathematical optimization

Geodesy

Machine learning

Benchmark (surveying)

Artificial intelligence

Geography

Estimation theory

Symbolic regression

Overhead (engineering)

Computer science

Non-linear least squares

Botany

Quantum mechanics

Genetic Programming and Evolvable Machines

In this paper we analyze the effects of using nonlinear least squares for parameter identification of symbolic regression models and integrate it as local search mechanism in tree-based genetic programming. We employ the Levenberg–Marquardt algorithm for parameter optimization and calculate gradients via automatic differentiation. We provide examples where the parameter identification succeeds and fails and highlight its computational overhead. Using an extensive suite of symbolic regression benchmark problems we demonstrate the increased performance when incorporating nonlinear least squares within genetic programming. Our results are compared with recently published results obtained by several genetic programming variants and state of the art machine learning algorithms. Genetic programming with nonlinear least squares performs among the best on the defined benchmark suite and the local search can be easily integrated in different genetic programming algorithms as long as only differentiable functions are used within the models.

Parameter identification for symbolic regression using nonlinear least squares