Pattern recognition (psychology)

Metric (unit)

Data mining

Machine learning

Engineering

Artificial intelligence

Backpropagation

Test data

Programming language

Sensitivity (control systems)

Operations management

Electronic engineering

Computer science

Artificial neural network

Applied Sciences

Accuracy metrics have been widely used for the evaluation of predictions in machine learning. However, the selection of an appropriate accuracy metric for the evaluation of a specific prediction has not yet been specified. In this study, seven of the most used accuracy metrics in machine learning were summarized, and both their advantages and disadvantages were studied. To achieve this, the acoustic emission data of damage locations were collected from a pile hit test. A backpropagation artificial neural network prediction model for damage locations was trained with acoustic emission data using six different training algorithms, and the prediction accuracies of six algorithms were evaluated using seven different accuracy metrics. Test results showed that the training algorithm of “TRAINGLM” exhibited the best performance for predicting damage locations in deep piles. Subsequently, the artificial neural networks were trained using three different datasets collected from three acoustic emission sensor groups, and the prediction accuracies of three models were evaluated with the seven different accuracy metrics. The test results showed that the dataset collected from the pile body-installed sensors group exhibited the highest accuracy for predicting damage locations in deep piles. Subsequently, the correlations between the seven accuracy metrics and the sensitivity of each accuracy metrics were discussed based on the analysis results. Eventually, a novel selection method for an appropriate accuracy metric to evaluate the accuracy of specific predictions was proposed. This novel method is useful to select an appropriate accuracy metric for wide predictions, especially in the engineering field.

Study on Accuracy Metrics for Evaluating the Predictions of Damage Locations in Deep Piles Using Artificial Neural Networks with Acoustic Emission Data