Bayesian probability

Perspective (graphical)

Pure mathematics

Mathematics

Computer science

Machine learning

Engineering

Bayesian inference

Selection (genetic algorithm)

Management science

Field (mathematics)

Categorization

Artificial intelligence

Data science

Frequentist inference

Ranking (information retrieval)

Operations research

Frontiers of Engineering Management

In this paper, we briefly review the development of ranking-and-selection (R&S) in the past 70 years, especially the theoretical achievements and practical applications in the last 20 years. Different from the frequentist and Bayesian classifications adopted by Kim and Nelson (2006b) and Chick (2006) in their review articles, we categorize existing R&S procedures into fixed-precision and fixed-budget procedures, as in Hunter and Nelson (2017). We show that these two categories of procedures essentially differ in the underlying methodological formulations, i.e., they are built on hypothesis testing and dynamic-programming, respectively. In light of this variation, we review in detail some well-known procedures in the literature and show how they fit into these two formulations. In addition, we discuss the use of R&S procedures in solving various practical problems and propose what we think are the important research questions in the field.

Review on ranking and selection: A new perspective