Behavioral and brain rhythms in the millisecond-to-second range are central in human music, speech, and movement. A comparative approach can further our understanding of the evolution of rhythm processing by identifying behavioral and neural similarities and differences across cognitive domains and across animal species. We provide an overview of research into rhythm cognition in music, speech, and animal communication. Rhythm has received considerable attention within each individual field, but to date, little integration. This review article on rhythm processing incorporates and extends existing ideas on temporal processing in speech and music and offers suggestions about the neural, biological, and evolutionary bases of human abilities in these domains.