Geodesy

Quantum mechanics

Random forest

Benchmark (surveying)

Ensemble learning

Biology

Identification (biology)

Semi-supervised learning

Statistics

Similarity (geometry)

Training (meteorology)

Weighting

Power (physics)

Data mining

Computer science

Botany

Radiology

Physics

Regression

Supervised learning

Mathematics

Geography

Image (mathematics)

Mathematical analysis

Reliability (semiconductor)

Artificial intelligence

Decision tree

Cluster analysis

Tree (set theory)

Medicine

Software engineering

Meteorology

Artificial neural network

Training set

Oracle

Machine learning

Knowledge-Based Systems

Semi-supervised learning (SSL) aims to use unlabeled data as an additional source of information in order to improve upon the performance of supervised learning methods. The availability of labeled data is often limited due to the expensive and/or tedious annotation process, while unlabeled data could be easily available in large amounts. This is particularly true for predictive modelling problems with a structured output space. In this study, we address the task of SSL for multi-target regression (MTR), where the output space consists of multiple numerical values. We extend the self-training approach to perform SSL for MTR by using a random forest of predictive clustering trees. In self-training, a model iteratively uses its own most reliable predictions, hence a good measure for the reliability of predictions is essential. Given that reliability estimates for MTR predictions have not yet been studied, we propose four such estimates, based on mechanisms provided within ensemble learning. In addition to these four scores, we use two benchmark scores (oracle and random) to empirically determine the performance limits of self-training. We also propose an approach to automatically select a threshold for the identification of the most reliable predictions to be used in the next iteration. An empirical evaluation on a large collection of datasets for MTR shows that self-training with any of the proposed reliability scores is able to consistently improve over supervised random forests and multi-output support vector regression. This is also true when the reliability threshold is selected automatically.

Self-training for multi-target regression with tree ensembles