High-entropy ceramics with five or more cations have recently attracted significant attention due to their superior properties for various structural and functional applications. Although the multi-component ceramics have been of interest for several decades, the concept of high-entropy ceramics was defined in 2004 by producing the first high-entropy nitride films. Following the introduction of the entropy stabilization concept, significant efforts were started to increase the entropy, minimize the Gibbs free energy and achieve stable single-phase high-entropy ceramics. High-entropy oxides, nitrides, carbides , borides and hydrides are currently the most popular high-entropy ceramics due to their potential for various applications, while the study of other ceramics, such as silicides , sulfides, fluorides, phosphides, phosphates, oxynitrides , carbonitrides and borocarbonitrides, is also growing fast. In this paper, the progress regarding high-entropy ceramics is reviewed from both experimental and theoretical points of view. Different aspects including the history, principles, compositions, crystal structure, theoretical/empirical design (via density functional theory, molecular dynamics simulation, machine learning, CALPHAD and descriptors), production methods and properties are thoroughly reviewed. The paper specifically attempts to answer how these materials with remarkable structures and properties can be used in future applications.