Pathology

Response bias

Board certification

Mathematics

Medical education

Family medicine

Statistics

Non-response bias

Political science

Psychology

Emergency medicine

Survey data collection

Continuing medical education

Continuing education

Certification

Rapid response team

Medicine

Social psychology

Evaluation & the Health Professions

For survey researchers, physicians in the United States are a difficult-to-reach subgroup. The purpose of this study is to quantify the effect of email reminders on web-based survey response rates targeting physicians. We conducted a retrospective analysis of 11 American Board of Internal Medicine surveys from 2017 to 2019. We compute aggregate response rates for the periods between weekly email contacts across the 11 surveys, while controlling for survey time to complete, physician age, gender, region, board certification status, and initial exam performance. The overall predicted response rate after six weekly email contacts was 23.7%, 95% CI: (17.1%, 33.0%). Across the 11 surveys, we found response rate for the first period to be 8.9%, 95% CI: (6.5%, 12.2%). We observed a 50% decrease in response from the first to the second period, which had a 4.4%, 95% CI: (3.2%, 6.2%), response rate. The third and fourth response periods yielded similar response rates of 3.0%, 95% CI: (2.3%, 3.9%) and 3.3%, 95%CI: (2.4%, 4.6%), respectively. The fifth and sixth response periods yielded similar response rates of 2.2%, 95%CI: (1.5%, 3.3%) and 1.9%, 95% CI: (1.3%, 2.7%), respectively. The results were further stratified into different levels of participant survey interest, and are helpful for cost and sample size considerations when designing a physician survey.

Remind Me Again: Physician Response to Web Surveys: The Effect of Email Reminders Across 11 Opinion Survey Efforts at the American Board of Internal Medicine from 2017 to 2019