Women are underrepresented in fields in which success is believed to require brilliance, but the reasons for this pattern are poorly understood. We investigated perceptions of a "masculinity-contest culture," an organizational environment of ruthless competition, as a key mechanism whereby a perceived emphasis on brilliance discourages female participation. Across three preregistered correlational and experimental studies involving adult lay participants online (<i>N</i> = 870) and academics from more than 30 disciplines (<i>N</i> = 1,347), we found a positive association between the perception that a field or an organization values brilliance and the perception that this field or organization is characterized by a masculinity-contest culture. This association was particularly strong among women. In turn, perceiving a masculinity-contest culture predicted lower interest and sense of belonging as well as stronger impostor feelings. Experimentally reducing the perception of a masculinity-contest culture eliminated gender gaps in interest and belonging in a brilliance-oriented organization, suggesting possible avenues for intervention.