Endoscopists' diagnostic accuracy in detecting upper gastrointestinal neoplasia in the framework of artificial intelligence studies.

Published on May 5, 2021in Endoscopy7.341
· DOI :10.1055/A-1500-3730
AIMS Estimates on miss-rates for upper gastrointestinal neoplasia (UGIN) rely on registry data or old studies. Quality assurance programs for upper-GI endoscopy are not fully established due to the lack of the infrastructure to measure endoscopists' competence. We aimed at assessing endoscopists' accuracy for the recognition of UGIN exploiting the framework of Artificial Intelligence (AI) validation studies. METHODS Literature search among databases (PubMed/MEDLINE, EMBASE, Scopus) up to August 2020 was performed to identify articles evaluating the accuracy of individual endoscopists for the recognition of UGIN within studies validating AI against a histologically-verified expert-annotated ground-truth. Main outcomes were endoscopists' pooled sensitivity, specificity, positive and negative predictive values (PPV/NPV), Area Under the Curve (AUCs) for all-UGIN, for esophageal squamous-cell neoplasia (ESCN), Barrett-related neoplasia (BERN), and gastric adenocarcinoma (GAC). RESULTS Seven studies with 122 endoscopists were included (2 ESCN, 3 BERN, 1 GAC, 1 UGIN overall). Pooled endoscopists sensitivity and specificity for UGIN was 82% (CI 80-84%) and 79% (CI 76-81%), respectively. Endoscopists' accuracy was higher for GAC detection (AUC 0.95, CI 0.93-0.98) than ESCN detection (AUC 0.90, CI 0.88-0.92) and BERN detection (AUC 0.86, CI0.84-0.88). Sensitivity was higher for Eastern vs. Western endoscopists (87%, CI 84-89% vs. 75%, CI 72-78%), and for experts vs. non-experts endoscopists (85%, CI 83-87% vs. 71%, CI 67-75%). CONCLUSION We show suboptimal endoscopists' accuracy for the recognition of UGIN even within a framework that included higher prevalence and disease awareness. Future AI validation studies represent a framework to assess endoscopist competence.
📖 Papers frequently viewed together
6 Authors (Yunxia Feng, ..., Dan Liao)
2 Citations
1 Citations
#1Julia Arribas (University of Porto)H-Index: 1
#2Giulio Antonelli (Sapienza University of Rome)H-Index: 13
Last. Mário Dinis-Ribeiro (University of Porto)H-Index: 57
view all 15 authors...
Objective Artificial intelligence (AI) may reduce underdiagnosed or overlooked upper GI (UGI) neoplastic and preneoplastic conditions, due to subtle appearance and low disease prevalence. Only disease-specific AI performances have been reported, generating uncertainty on its clinical value. Design We searched PubMed, Embase and Scopus until July 2020, for studies on the diagnostic performance of AI in detection and characterisation of UGI lesions. Primary outcomes were pooled diagnostic accuracy...
12 CitationsSource
#1Yohei Ikenoyama (JFCR: Japanese Foundation for Cancer Research)H-Index: 3
#2Toshiaki Hirasawa (JFCR: Japanese Foundation for Cancer Research)H-Index: 24
Last. Tomohiro TadaH-Index: 20
view all 14 authors...
OBJECTIVES: Detecting early gastric cancer is difficult, and it may even be overlooked by experienced endoscopists. Recently, artificial intelligence based on deep learning through convolutional neural networks (CNNs) has enabled significant advancements in the field of gastroenterology. However, it remains unclear whether a CNN can outperform endoscopists. In this study, we evaluated whether the performance of a CNN in detecting early gastric cancer is better than that of endoscopists. METHODS:...
20 CitationsSource
#1Hiromu FukudaH-Index: 6
#2Ryu IshiharaH-Index: 35
Last. Tomohiro Tada (UTokyo: University of Tokyo)H-Index: 20
view all 10 authors...
ABSTRACT Background and Aims Narrow-band imaging (NBI) is currently regarded as the standard modality for diagnosing esophageal squamous cell carcinoma (SCC). We developed a computerized image-analysis system for diagnosing esophageal SCC by NBI and estimated its performance with video images. Methods Altogether, 23,746 images from 1544 pathologically proven superficial esophageal SCCs and 4587 images from 458 noncancerous and normal tissue were used for construction of Artificial Intelligence (...
20 CitationsSource
#1Melina Arnold (IARC: International Agency for Research on Cancer)H-Index: 24
#2Christian C. AbnetH-Index: 93
Last. Freddie Bray (IARC: International Agency for Research on Cancer)H-Index: 119
view all 7 authors...
Abstract Background & Aims There were an estimated 4.8 million new cases of gastrointestinal (GI) cancers and 3.4 million related deaths, worldwide, in 2018. GI cancers account for 26% of the global cancer incidence and 35% of all cancer-related deaths. We investigated the global burden from the 5 major GI cancers as well as geographic and temporal trends in cancer-specific incidence and mortality. Methods Data on primary cancers of the esophagus, stomach, colorectum, liver, and pancreas were ex...
139 CitationsSource
#1Fons van der SommenH-Index: 12
#2Jeroen de GroofH-Index: 3
Last. Jacques J. BergmanH-Index: 67
view all 11 authors...
There has been a vast increase in GI literature focused on the use of machine learning in endoscopy. The relative novelty of this field poses a challenge for reviewers and readers of GI journals. To appreciate scientific quality and novelty of machine learning studies, understanding of the technical basis and commonly used techniques is required. Clinicians often lack this technical background, while machine learning experts may be unfamiliar with clinical relevance and implications for daily pr...
25 CitationsSource
#1Yusuke HoriuchiH-Index: 13
#2Kazuharu Aoyama (Arai Helmet)H-Index: 14
Last. Tomohiro TadaH-Index: 20
view all 10 authors...
Background Early detection of early gastric cancer (EGC) allows for less invasive cancer treatment. However, differentiating EGC from gastritis remains challenging. Although magnifying endoscopy with narrow band imaging (ME-NBI) is useful for differentiating EGC from gastritis, this skill takes substantial effort. Since the development of the ability to convolve the image while maintaining the characteristics of the input image (convolution neural network: CNN), allowing the classification of th...
38 CitationsSource
#1Albert J. de Groof (UvA: University of Amsterdam)H-Index: 5
#2Maarten R. Struyvenberg (UvA: University of Amsterdam)H-Index: 7
Last. Jacques J. Bergman (UvA: University of Amsterdam)H-Index: 67
view all 18 authors...
Abstract Background & Aims We aimed to develop and validate a deep-learning computer-aided detection (CAD) system, suitable for use in real time in clinical practice, to improve endoscopic detection of early neoplasia in patients with Barrett’s esophagus (BE). Methods We developed a hybrid ResNet-UNet model CAD system using 5 independent endoscopy datasets. We performed pre-training using 494,364 labelled endoscopic images collected from all intestinal segments. Then, we used 1704 unique esophag...
91 CitationsSource
#1Adrian Saftoiu (University of Medicine and Pharmacy of Craiova)H-Index: 29
#2Cesare HassanH-Index: 92
Last. Thierry PonchonH-Index: 56
view all 18 authors...
In Europe at present, but also in 2040, 1 in 3 cancer-related deaths are expected to be caused by digestive cancers. Endoscopic technologies enable diagnosis, with relatively low invasiveness, of precancerous conditions and early cancers, thereby improving patient survival. Overall, endoscopy capacity must be adjusted to facilitate both effective screening programs and rigorous control of the quality assurance and surveillance systems required. 1 For average-risk populations, ESGE recommends the...
29 CitationsSource
#1Masayasu Ohmori (Okayama University)H-Index: 8
#2Ryu IshiharaH-Index: 35
Last. Tomohiro Tada (UTokyo: University of Tokyo)H-Index: 20
view all 14 authors...
ABSTRACT Background and Aims Diagnosing esophageal squamous cell carcinoma (SCC) depends on individual physician expertise and may be subject to interobserver variability. Therefore, we developed a computerized image-analysis system to detect and differentiate esophageal SCC. Methods A total of 9591 nonmagnified endoscopic (ME) and 7844 ME images of pathologically confirmed superficial esophageal SCCs and 1692 non-ME and 3435 ME images from noncancerous lesions or normal esophagus were used as t...
44 CitationsSource
#1Huiyan Luo (SYSU: Sun Yat-sen University)H-Index: 10
#2Guo Liang Xu (SYSU: Sun Yat-sen University)H-Index: 14
Last. Rui-Hua Xu (SYSU: Sun Yat-sen University)H-Index: 67
view all 26 authors...
Summary Background Upper gastrointestinal cancers (including oesophageal cancer and gastric cancer) are the most common cancers worldwide. Artificial intelligence platforms using deep learning algorithms have made remarkable progress in medical imaging but their application in upper gastrointestinal cancers has been limited. We aimed to develop and validate the Gastrointestinal Artificial Intelligence Diagnostic System (GRAIDS) for the diagnosis of upper gastrointestinal cancers through analysis...
101 CitationsSource
Cited By0