Quantifying idiosyncratic and shared contributions to judgment

Identifying relative idiosyncratic and shared contributions to judgments is a fundamental challenge to the study of human behavior, yet there is no established method for estimating these contributions. Using edge cases of stimuli varying in intrarater reliability and interrater agreement—faces (high on both), objects (high on the former, low on the latter), and complex patterns (low on both)—we showed that variance component analyses (VCAs)...