With a day-side temperature in excess of 4500K, comparable to a mid-K-type star, KELT-9b is the hottest planet known. Its extreme temperature makes KELT-9b a particularly interesting test bed for investigating the nature and diversity of gas giant planets. We observed the transit of KELT-9b at high spectral resolution (R$\sim$94,600) with the CARMENES instrument on the Calar Alto 3.5-m telescope. Using these data, we detect for the first time ionized calcium (CaII triplet) absorption in the atmosphere of KELT-9b; this is the second time that CaII has been observed in a hot Jupiter. Our observations also reveal prominent H$\alpha$ absorption, confirming the presence of an extended hydrogen envelope around KELT-9b. We compare our detections with an atmospheric model and find that all four lines form between atmospheric temperatures of 6100 K and 8000 K and that the CaII lines form at pressures between 10 and 50 nbar while the H$\alpha$ line forms at a lower pressure ($\sim$6 nbar), higher up in the atmosphere. The altitude that the core of H$\alpha$ line forms is found to be $\sim$1.4 R$_{p}$, well within the planetary Roche lobe ($\sim$1.9 R$_{p}$). Therefore, rather than probing the escaping upper atmosphere directly, the H$\alpha$ line and the other observed Balmer and metal lines serve as atmospheric thermometers enabling us to probe the planet's temperature profile, thus energy budget.