Algorithm

Linguistics

Philosophy

Random variable

Discrete mathematics

Statistics

Distribution (mathematics)

Computer science

Physics

Divergence (linguistics)

Database

Mathematics

Mathematical analysis

Interval (graph theory)

Infimum and supremum

Measure (data warehouse)

Combinatorics

Multivariate random variable

IEEE Transactions on Information Theory

The random variable simulation problem consists in using a k-dimensional i.i.d. random vector X k with distribution P X k to simulate an n-dimensional i.i.d. random vector Y n so that its distribution is approximately Q Y n . In contrast to previous works, in this paper, we consider the standard Rényi divergence and two variants of all orders to measure the level of approximation. These two variants are the max-Rényi divergence D α max (P, Q) and the sum-Rényi divergence Du (P, Q). When α = ∞, these two measures are strong because for any ϵ ≥ 0, D ∞ max (P, Q) ≤ ϵ or D ∞ + (P, Q) ≤ ϵ implies e -ϵ ≤ (P(x)/Q(x)) <; e ϵ for all x. Under these Rényi divergence measures, we characterize the asymptotics of normalized divergences as well as the Rényi conversion rates. The latter is defined as the supremum of n/k such that the Rényi divergences vanish asymptotically. Our results show that, when the Rényi parameter is in the interval (0, 1), the Rényi conversion rates equal the ratio of the Shannon entropies H (P X )/H (Q Y ), which is consistent with traditional results in which the total variation measure was adopted. When the Rényi parameter is in the interval (1, ∞), the Rényi conversion rates are, in general, smaller than H (P X )/H (Q Y ). When specialized to the case in which either P X or Q Y is uniform, the simulation problem reduces to the source resolvability and intrinsic randomness problems. The preceding results are used to characterize the asymptotics of Rényi divergences and the Rényi conversion rates for these two cases.

Simulation of Random Variables Under Rényi Divergence Measures of All Orders