Reliable Accuracy Estimates from k-Fold Cross Validation

It is popular to evaluate the performance of classification algorithms by k-fold cross validation. A reliable accuracy estimate will have a relatively small variance, and several studies therefore suggested to repeatedly perform k-fold cross validation. Most of them did not consider the correlation among the replications of k-fold cross validation, and hence the variance could be underestimated. The purpose of this study is to explore whether...