The ages, metallicities, alpha-elements, and integrals of motion of globular clusters (GCs) accreted by the Milky Way from disrupted satellites remain largely unchanged over time. Here we have used these conserved properties in combination to assign 76 GCs to five progenitor satellite galaxies – one of which we dub the Koala dwarf galaxy. We fit a leaky-box chemical enrichment model to the age–metallicity distribution of GCs, deriving the effective yield and the formation epoch of each satellite. Based on scaling relations of GC counts we estimate the original halo mass, stellar mass, and mean metallicity of each satellite. The total stellar mass of the five accreted satellites contributed around 109 M⊙ in stars to the growth of the Milky Way but over 50 per cent of the Milky Way’s GC system. The five satellites formed at very early times and were likely accreted 8–11 Gyr ago, indicating rapid growth for the Milky Way in its early evolution. We suggest that at least three satellites were originally nucleated, with the remnant nucleus now a GC of the Milky Way. 11 GCs are also identified as having formed ex situ but could not be assigned to a single progenitor satellite.