We present a detailed study of the structural dynamics, energetic and dynamical stability and thermal transport of the bismuth chalcogenides Bi2S3, Bi2Se3 and Bi2Te3 and their alloys. The differing activities of the Bi cation lone pairs in Bi2S3, Bi2Se3 and Bi2Te3 lead to competition between orthorhombic Pnma and rhombohedral R-3m phases, with the latter favored by heavier chalcogen atoms, while the reported non-ambient phases of Bi2Se3 and Bi2Te3 show phonon instabilities under ambient conditio...