Geodesy

Algorithm

Benchmark (surveying)

Philosophy

Statistics

Selection (genetic algorithm)

Computer science

Mathematical optimization

Random search

Mathematics

Geography

Monte Carlo tree search

Epistemology

Mathematical analysis

Artificial intelligence

Tree (set theory)

Monte Carlo method

Deep learning

Artificial neural network

Model selection

Property (philosophy)

Machine learning

Information Sciences

The selection of hyper-parameters in building neural networks is often left to end-users. However, this may lead to poor performance regardless of the amount of experience of the user. With deep implementation, this becomes even more challenging due to a large number of hyper-parameters. Deep stochastic configuration networks (DeepSCNs) have become increasingly popular over the past years because of their universal approximation property, fast learning and easy implementation. So far, understanding and setting the hyper-parameters for this type of network is still unexplored. This paper defines a suitable search space for DeepSCN, a performance estimation strategy for finding suitable hyper-parameters with search strategies Monte-Carlo tree search (MCTS) and random search. Simulations are performed using both searches over four benchmark datasets, and results indicate some significant improvements in modelling performance. Furthermore, a case study is presented to demonstrate that an optimised model and hyper-parameters can be found using MCTS.

Deep stochastic configuration networks with optimised model and hyper-parameters