Overfitting

Mathematics

Human–computer interaction

Cloning (programming)

Geodesy

Machine learning

Mathematical analysis

Benchmark (surveying)

Leverage (statistics)

Artificial intelligence

Geography

Generalization

Programming language

Computer science

Theoretical computer science

Scalability

Artificial neural network

Database

Driving requires reacting to a wide variety of complex environment conditions and agent behaviors. Explicitly modeling each possible scenario is unrealistic. In contrast, imitation learning can, in theory, leverage data from large fleets of human-driven cars. Behavior cloning in particular has been successfully used to learn simple visuomotor policies end-to-end, but scaling to the full spectrum of driving behaviors remains an unsolved problem. In this paper, we propose a new benchmark to experimentally investigate the scalability and limitations of behavior cloning. We show that behavior cloning leads to state-ofthe-art results, executing complex lateral and longitudinal maneuvers, even in unseen environments, without being explicitly programmed to do so. However, we confirm some limitations of the behavior cloning approach: some wellknown limitations (e.g., dataset bias and overfitting), new generalization issues (e.g., dynamic objects and the lack of a causal modeling), and training instabilities, all requiring further research before behavior cloning can graduate to real-world driving. The code, dataset, benchmark, and agent studied in this paper can be found at http:// github.com/felipecode/coiltraine/blob/ master/docs/exploring_limitations.md.

Exploring the Limitations of Behavior Cloning for Autonomous Driving