Farmers in Africa have long adapted to climatic and other risks by diversifying their farming activities. Using a multi-scale approach, we explore the relationship between farming diversity and food security and the diversification potential of African agriculture and its limits on the household and continental scale. On the household scale, we use agricultural surveys from more than 28,000 households located in 18 African countries. In a next step, we use the relationship between rainfall, rainfall variability, and farming diversity to determine the available diversification options for farmers on the continental scale. On the household scale, we show that households with greater farming diversity are more successful in meeting their consumption needs, but only up to a certain level of diversity per ha cropland and more often if food can be purchased from off-farm income or income from farm sales. More diverse farming systems can contribute to household food security; however, the relationship is influenced by other factors, for example, the market orientation of a household, livestock ownership, nonagricultural employment opportunities, and available land resources. On the continental scale, the greatest opportunities for diversification of food crops, cash crops, and livestock are located in areas with 500-1,000 mm annual rainfall and 17%-22% rainfall variability. Forty-three percent of the African cropland lacks these opportunities at present which may hamper the ability of agricultural systems to respond to climate change. While sustainable intensification practices that increase yields have received most attention to date, our study suggests that a shift in the research and policy paradigm toward agricultural diversification options may be necessary.