Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices

Published on Aug 28, 2018
· DOI :10.1038/S41746-018-0040-6
Michael D. Abràmoff66
Estimated H-index: 66
Philip T. Lavin1
Estimated H-index: 1
+ 2 AuthorsJames C. Folk24
Estimated H-index: 24
(UI: University of Iowa)
Artificial Intelligence (AI) has long promised to increase healthcare affordability, quality and accessibility but FDA, until recently, had never authorized an autonomous AI diagnostic system. This pivotal trial of an AI system to detect diabetic retinopathy (DR) in people with diabetes enrolled 900 subjects, with no history of DR at primary care clinics, by comparing to Wisconsin Fundus Photograph Reading Center (FPRC) widefield stereoscopic photography and macular Optical Coherence Tomography (OCT), by FPRC certified photographers, and FPRC grading of Early Treatment Diabetic Retinopathy Study Severity Scale (ETDRS) and Diabetic Macular Edema (DME). More than mild DR (mtmDR) was defined as ETDRS level 35 or higher, and/or DME, in at least one eye. AI system operators underwent a standardized training protocol before study start. Median age was 59 years (range, 22–84 years); among participants, 47.5% of participants were male; 16.1% were Hispanic, 83.3% not Hispanic; 28.6% African American and 63.4% were not; 198 (23.8%) had mtmDR. The AI system exceeded all pre-specified superiority endpoints at sensitivity of 87.2% (95% CI, 81.8–91.2%) (>85%), specificity of 90.7% (95% CI, 88.3–92.7%) (>82.5%), and imageability rate of 96.1% (95% CI, 94.6–97.3%), demonstrating AI’s ability to bring specialty-level diagnostics to primary care settings. Based on these results, FDA authorized the system for use by health care providers to detect more than mild DR and diabetic macular edema, making it, the first FDA authorized autonomous AI diagnostic system in any field of medicine, with the potential to help prevent vision loss in thousands of people with diabetes annually. ClinicalTrials.gov NCT02963441
Figures & Tables
📖 Papers frequently viewed together
15 Authors (Varun Gulshan, ..., Dale R. Webster)
2,401 Citations
318 Citations
7 Authors (Andre Esteva, ..., Sebastian Thrun)
3,982 Citations
#1Abhay ShahH-Index: 13
#2Stephanie K. Lynch (UI: University of Iowa)H-Index: 4
Last. Michael D. AbràmoffH-Index: 66
view all 9 authors...
Deep learning algorithms, typically implemented as Convolutional Neural Networks (CNNs), in recent years have gained traction in medical image analysis. The majority of CNNs employed in retinal image diagnosis applications are image-based; wherein input is the retinal image and output is the classification/diagnosis, resulting in a blackbox like algorithm. In contrast, hybrid lesion-based algorithms employ multiple CNN-based detectors to categorically detect various lesions in the image, and fin...
7 CitationsSource
#1Stephanie K. Lynch (UI: University of Iowa)H-Index: 4
Abstract Since 1875, controversy has ensued over whether ocular diabetic complications are primarily vasculopathic or neuropathic in nature. Here, we discuss the historical context by which diabetic retinopathy (DR) came to be considered a primary vasculopathy, in contrast to more recent data suggesting the importance of diabetic retinal neurodegeneration (DRN) as the primary manifestation of ocular diabetic damage. Unsurprisingly, DRN parallels other diabetic complications related to neuropathy...
50 CitationsSource
#1Varun Gulshan (Google)H-Index: 8
#2Lily Peng (Google)H-Index: 32
Last. Dale R. Webster (Google)H-Index: 22
view all 15 authors...
Importance Deep learning is a family of computational methods that allow an algorithm to program itself by learning from a large set of examples that demonstrate the desired behavior, removing the need to specify rules explicitly. Application of these methods to medical imaging requires further assessment and validation. Objective To apply deep learning to create an algorithm for automated detection of diabetic retinopathy and diabetic macular edema in retinal fundus photographs. Design and Sett...
2,401 CitationsSource
#1Michael D. Abràmoff (UIHC: University of Iowa Hospitals and Clinics)H-Index: 66
#2Yiyue Lou (UI: University of Iowa)H-Index: 8
Last. Meindert NiemeijerH-Index: 35
view all 7 authors...
To compare performance of a deep-learning enhanced algorithm for automated detection of diabetic retinopathy (DR), to the previously published performance of that algorithm, the Iowa Detection Program (IDP)-without deep learning components-on the same publicly available set of fundus images and previously reported consensus reference standard set, by three US Board certified retinal specialists.We used the previously reported consensus reference standard of referable DR (rDR), defined as Interna...
332 CitationsSource
#1Roger Chou (OHSU: Oregon Health & Science University)H-Index: 84
#2Tracy Dana (OHSU: Oregon Health & Science University)H-Index: 29
Last. Ian Blazina (OHSU: Oregon Health & Science University)H-Index: 19
view all 5 authors...
Importance Impaired visual acuity is common among older adults and can adversely affect function and quality of life. Objective To update a 2009 systematic review on screening for impaired visual acuity among older adults for the US Preventive Services Task Force (USPSTF). Data Sources Ovid MEDLINE (2008 to January 2016), Cochrane Central Register of Controlled Trials, and Cochrane Database of Systematic Reviews. Study Selection Randomized clinical trials of screening; diagnostic accuracy studie...
42 CitationsSource
#1Yu T. Wang (JHUSOM: Johns Hopkins University School of Medicine)H-Index: 1
#2Mongkol Tadarati (Rangsit University)H-Index: 7
Last. Neil M. Bressler (JHUSOM: Johns Hopkins University School of Medicine)H-Index: 93
view all 5 authors...
Importance Diagnosing diabetic macular edema (DME) from monocular fundus photography vs optical coherence tomography (OCT) central subfield thickness (CST) can yield different prevalence rates for DME. Epidemiologic studies and telemedicine screening typically use monocular fundus photography, while treatment of DME uses OCT CST. Objective To compare DME prevalence from monocular fundus photography and OCT. Design, Setting, and Participants Retrospective cross-sectional study of DME grading base...
40 CitationsSource
#1Isabel N. Figueiredo (UC: University of Coimbra)H-Index: 15
#2Sunil Kumar (National Institute of Technology Delhi)H-Index: 21
Last. Björn Engquist (University of Texas at Austin)H-Index: 52
view all 5 authors...
Diabetic retinopathy (DR) is a sight-threatening condition occurring in persons with diabetes, which causes progressive damage to the retina. The early detection and diagnosis of DR is vital for saving the vision of diabetic persons. The early signs of DR which appear on the surface of the retina are the dark lesions such as microaneurysms (MAs) and hemorrhages (HEMs), and bright lesions (BLs) such as exudates. In this paper, we propose a novel automated system for the detection and diagnosis of...
38 CitationsSource
Background The worldwide epidemic of type 2 diabetes requires effective prevention. We determined the long-term extent of beneficial effects of lifestyle intervention and metformin on diabetes prevention, originally demonstrated during the 3-year Diabetes Prevention Program (DPP), and whether diabetes-associated microvascular complications are reduced.
384 CitationsSource
#1Morten Balle Hansen (University of Southern Denmark)H-Index: 14
#2Michael D. Abràmoff (UI: University of Iowa)H-Index: 66
Last. Tunde Peto (Moorfields Eye Hospital)H-Index: 49
view all 6 authors...
Objective Digital retinal imaging is an established method of screening for diabetic retinopathy (DR). It has been established that currently about 1% of the world’s blind or visually impaired is due to DR. However, the increasing prevalence of diabetes mellitus and DR is creating an increased workload on those with expertise in grading retinal images. Safe and reliable automated analysis of retinal images may support screening services worldwide. This study aimed to compare the Iowa Detection P...
34 CitationsSource
#2John A. WellsH-Index: 21
#3Adam R. GlassmanH-Index: 47
Last. Roy W. BeckH-Index: 116
view all 17 authors...
Purpose: To provide 2-year results comparing antievascular endothelial growth factor (VEGF) agents for center-involved diabetic macular edema (DME) using a standardized follow-up and retreatment regimen. Design: Randomized clinical trial. Participants: Six hundred sixty participants with visual acuity (VA) impairment from DME. Methods: Randomization to 2.0-mg aflibercept, 1.25-mg repackaged (compounded) bevacizumab, or 0.3mg ranibizumab intravitreous injections performed up to monthly using a pr...
770 CitationsSource
Cited By161
#1Ryung Lee (Touro College of Osteopathic Medicine)
#1Weisha Wang (University of Southampton)
#2Long Chen (University of Southampton)
Last. Yichuan Wang (University of Sheffield)H-Index: 22
view all 0 authors...
Artificial Intelligence (AI) technology is transforming the healthcare sector. However, despite this, the associated ethical implications remain open to debate. This research investigates how signals of AI responsibility impact healthcare practitioners’ attitudes toward AI, satisfaction with AI, AI usage intentions, including the underlying mechanisms. Our research outlines autonomy, beneficence, explainability, justice, and non-maleficence as the five key signals of AI responsibility for health...
#1Nikos Tsiknakis (FORTH: Foundation for Research & Technology – Hellas)
#2Dimitris Theodoropoulos (Mediterranean University)
Last. Kostas Marias (FORTH: Foundation for Research & Technology – Hellas)H-Index: 24
view all 0 authors...
Abstract null null Diabetic Retinopathy is a retina disease caused by diabetes mellitus and it is the leading cause of blindness globally. Early detection and treatment are necessary in order to delay or avoid vision deterioration and vision loss. To that end, many artificial-intelligence-powered methods have been proposed by the research community for the detection and classification of diabetic retinopathy on fundus retina images. This review article provides a thorough analysis of the use of ...
#1Janusz Pieczynski (University of Warmia and Mazury in Olsztyn)H-Index: 3
#2Patrycja Kuklo (University of Warmia and Mazury in Olsztyn)H-Index: 3
Last. Andrzej Grzybowski (University of Warmia and Mazury in Olsztyn)H-Index: 33
view all 0 authors...
In the presence of the ever-increasing incidence of diabetes mellitus (DM), the prevalence of diabetic eye disease (DED) is also growing. Despite many improvements in diabetic care, DM remains a leading cause of visual impairment in working-age patients. So far, prevention has been the best way to protect vision. The sooner we diagnose DED, the more effective the treatment is. Thus, diabetic retinopathy (DR) screening, especially with imaging techniques, is a method of choice for vision protecti...
#1Ali H. Al-Timemy (UOB: University of Baghdad)H-Index: 12
#2Nebras H. Ghaeb (UOB: University of Baghdad)H-Index: 2
Last. Javier Escudero (Edin.: University of Edinburgh)H-Index: 30
view all 0 authors...
Clinical keratoconus (KCN) detection is a challenging and time-consuming task. In the diagnosis process, ophthalmologists must revise demographic and clinical ophthalmic examinations. The latter include slit-lamb, corneal topographic maps, and Pentacam indices (PI). We propose an Ensemble of Deep Transfer Learning (EDTL) based on corneal topographic maps. We consider four pretrained networks, SqueezeNet (SqN), AlexNet (AN), ShuffleNet (SfN), and MobileNet-v2 (MN), and fine-tune them on a dataset...
Peripheral artery disease is an atherosclerotic disorder which, when present, portends poor patient outcomes. Low diagnosis rates perpetuate poor management, leading to limb loss and excess rates of cardiovascular morbidity and death. Machine learning algorithms and artificially intelligent systems have shown great promise in application to many areas in health care, such as accurately detecting disease, predicting patient outcomes, and automating image interpretation. Although the application o...
1 CitationsSource
#1Lorenz Adlung (Weizmann Institute of Science)H-Index: 8
#2Yotam Cohen (Weizmann Institute of Science)H-Index: 5
Last. Eran Elinav (Weizmann Institute of Science)H-Index: 72
view all 4 authors...
Summary Machine learning is increasingly integrated into clinical practice, with applications ranging from pre-clinical data processing, bedside diagnosis assistance, patient stratification, treatment decision making, and early warning as part of primary and secondary prevention. However, a multitude of technological, medical, and ethical considerations are critical in machine-learning utilization, including the necessity for careful validation of machine-learning-based technologies in real-life...
Convolutional Neural Networks (CNNs) can play a key role in Medical Image Analysis under large-scale annotated datasets. However, preparing such massive dataset is demanding. In this context, Generative Adversarial Networks (GANs) can generate realistic but novel samples, and thus effectively cover the real image distribution. In terms of interpolation, the GAN-based medical image augmentation is reliable because medical modalities can display the human body's strong anatomical consistency at fi...