Evaluating the impacts of habitat loss and fragmentation is essential for biodiversity conservation. However, an understanding of how habitat loss and fragmentation interact and affect biodiversity is limited. Based on 134 landscapes in the agro-pastoral ecotone of northern China selected by a stratified random sampling approach, we used a structural equation model to evaluate the direct effects of habitat loss and its indirect effects (via habitat fragmentation, including edge density, patch density, mean patch area, and mean nearest-neighbor distance) on grassland plant species richness (including specialists and weeds) at the landscape scale. Our results show that habitat loss had a direct negative effect on overall species richness and grassland specialist richness but a positive effect on weed richness. Meanwhile, the increase in patch density caused by habitat loss had a positive effect on overall species richness and grassland specialist richness. These results indicate that the effects of habitat loss and fragmentation can differ for habitat specialists and generalists. Increasing the number of habitat patches in the landscape could increase the species richness of habitat specialists. Our findings emphasize the importance of the management of adding habitat patches in fragmented landscapes for conserving habitat specialists.