Political scientists who study the interplay between personality and politics overwhelmingly rely on short personality scales. We explore whether the length of the employed personality scales affects the criterion validity of the scales. We show that need for cognition (NfC) increases reliance on party cues, but only when a longer measure is employed. Additionally, while NfC increases reliance on policy information, the effect is more than twice as large when a longer measure is used. Finally, Big Five personality traits that have been dismissed as irrelevant to political ideology yield stronger and more consistent associations when larger batteries are employed. We also show that using high Cronbach’s alpha and factor loadings as indicators of scale quality does not improve the criterion validity of brief measures. Hence, the measurement of personality conditions the conclusions we draw about the role of personality in politics.