Evaluation of Prediction Models for Identifying Malignancy in Pulmonary Nodules Detected via Low-Dose Computed Tomography

Published on Feb 5, 2020
· DOI :10.1001/JAMANETWORKOPEN.2019.21221
Sandra González Maldonado5
Estimated H-index: 5
(DKFZ: German Cancer Research Center),
Stefan Delorme63
Estimated H-index: 63
(DKFZ: German Cancer Research Center)
+ 4 AuthorsRudolf Kaaks151
Estimated H-index: 151
(DKFZ: German Cancer Research Center)
Importance Malignancy prediction models based on participant-related characteristics and imaging parameters from low-dose computed tomography (CT) may improve decision-making regarding nodule management and diagnosis in lung cancer screening. Objective To externally validate 5 malignancy prediction models that were developed in screening settings, compared with 3 models that were developed in clinical settings, in terms of discrimination and absolute risk calibration among participants in the German Lung Cancer Screening Intervention trial. Design, Setting, and Participants In this population-based diagnostic study, malignancy probabilities were estimated by applying 8 prediction models to data from 1159 participants in the intervention arm of the Lung Cancer Screening Intervention trial, a randomized clinical trial conducted from October 23, 2007, to April 30, 2016, with ongoing follow-up. This analysis considers end points up to 1 year after individuals’ last screening visit. Inclusion criteria for participants were at least 1 noncalcified pulmonary nodule detected on any of 5 annual screening visits, receiving a lung cancer diagnosis within the active screening phase of the Lung Cancer Screening Intervention trial, and an unequivocal identification of the malignant nodules. Data analysis was performed from February 1, 2019, through December 5, 2019. Interventions Five annual rounds of low-dose multislice CT. Main Outcomes and Measures Discrimination ability and calibration of malignancy probabilities estimated by 5 models developed in data from screening studies (4 Pan-Canadian Early Detection of Lung Cancer Study [PanCan] models using a parsimonious approach including nodule spiculation [PanCan-1b] or a comprehensive approach including nodule spiculation [PanCan-2b], and PanCan-2b replacing the nodule diameter variable with mean diameter [PanCan-MD] or volume [PanCan-VOL], as well as a model developed by the UK Lung Cancer Screening trial) and 3 models developed in clinical settings (US Department of Veterans Affairs, Mayo Clinic, and Peking University People’s Hospital). Results A total of 1159 participants (median [range] age, 57.63 [50.34-71.89] years; 763 [65.8%] men) with 3903 pulmonary nodules were included in this study. For nodules detected in the prevalence round of CT, the PanCan models showed excellent discrimination (PanCan-1b: area under the curve [AUC], 0.93 [95% CI, 0.87-0.99]; PanCan-2b: AUC, 0.94 [95% CI, 0.89-0.99]; PanCan-MD: AUC, 0.94 [95% CI, 0.91-0.98]; PanCan-VOL: AUC, 0.94 [95% CI, 0.90-0.98]), and all of the screening models except PanCan-MD and PanCan-VOL showed acceptable calibration (PanCan-1b: Spiegelhalterz = −1.081;P = .28; PanCan-2b: Spiegelhalterz = 0.436;P = .67; PanCan-MD: Spiegelhalterz = 3.888;P  Conclusions and Relevance This diagnostic study found that PanCan models showed excellent discrimination and calibration in prevalence screenings, confirming their ability to improve nodule management in screening settings, although calibration to nodules detected in follow-up scans should be improved. The models developed by the Mayo Clinic, Peking University People’s Hospital, Department of Veterans Affairs, and UK Lung Cancer Screening Trial did not perform as well.
📖 Papers frequently viewed together
648 Citations
9 Citations
#1Peng Huang (Johns Hopkins University)H-Index: 103
#2Cheng Ting Lin (Johns Hopkins University)H-Index: 13
Last. Stephen Lam (UBC: University of British Columbia)H-Index: 86
view all 20 authors...
Summary Background Current lung cancer screening guidelines use either mean diameter, volume, or density of the largest lung nodule on the previous CT scan or appearance of a new nodule to ascertain the timing of the next CT scan. We aimed to develop an accurate screening protocol by estimating the 3-year lung cancer risk after two screening CT scans using deep learning of radiologists' CT readings and other universally available clinical information. Methods A deep learning algorithm (referred ...
30 CitationsSource
#1Ugo PastorinoH-Index: 70
#1Ugo PastorinoH-Index: 27
Last. Alfonso MarchianòH-Index: 31
view all 10 authors...
ABSTRACT Background The National Lung Screening Trial showed that lung cancer (LC) screening by three annual rounds of low-dose computed tomography (LDCT) reduces LC mortality. We evaluated the benefit of prolonged LDCT screening beyond 5years, and its impact on overall and LC specific mortality at 10years. Design The Multicentric Italian Lung Detection (MILD) trial prospectively randomized 4099 participants, to a screening arm (n=2376), with further randomization to annual (n=1190) or biennial ...
126 CitationsSource
#1Michael W. Marcus (University of Liverpool)H-Index: 15
#2Stephen W. DuffyH-Index: 111
Last. John K. Field (University of Liverpool)H-Index: 86
view all 7 authors...
Background Estimation of the clinical probability of malignancy in patients with pulmonary nodules will facilitate early diagnosis, determine optimum patient management strategies and reduce overall costs. Methods Data from the UK Lung Cancer Screening trial were analysed. Multivariable logistic regression models were used to identify independent predictors and to develop a parsimonious model to estimate the probability of lung cancer in lung nodules detected at baseline and at 3-month and 12-mo...
9 CitationsSource
#1Martin C. Tammemagi (Brock University)H-Index: 29
#2Alex J. Ritchie (Royal Brisbane and Women's Hospital)H-Index: 1
Last. Stephen LamH-Index: 86
view all 23 authors...
Abstract Objective In lung cancer screening practice low-dose computed tomography, diameter, and volumetric measurement have been used in the management of screen-detected lung nodules. The aim of this study was to compare the performance of nodule malignancy risk prediction tools using diameter or volume and between computer-aided detection (CAD) and radiologist measurements. Methods Multivariable logistic regression models were prepared by using data from two multicenter lung cancer screening ...
15 CitationsSource
#1Linda M. O’Keeffe (UoB: University of Bristol)H-Index: 13
#2Gemma M J Taylor (UoB: University of Bristol)H-Index: 11
Last. Sanne A.E. Peters (University of Oxford)H-Index: 41
view all 6 authors...
Objectives To investigate the sex-specific association between smoking and lung cancer. Design Systematic review and meta-analysis. Data sources We searched PubMed and EMBASE from 1 January 1999 to 15 April 2016 for cohort studies. Cohort studies before 1 January 1999 were retrieved from a previous meta-analysis. Individual participant data from three sources were also available to supplement analyses of published literature. Eligibility criteria for selecting studies Cohort studies reporting th...
71 CitationsSource
#1Kaman Chung (Radboud University Nijmegen)H-Index: 9
#2Onno M. Mets (UU: Utrecht University)H-Index: 18
view all 10 authors...
Objective To assess the performance of the Brock malignancy risk model for pulmonary nodules detected in routine clinical setting. Methods In two academic centres in the Netherlands, we established a list of patients aged ≥40 years who received a chest CT scan between 2004 and 2012, resulting in 16 850 and 23 454 eligible subjects. Subsequent diagnosis of lung cancer until the end of 2014 was established through linking with the National Cancer Registry. A nested case–control study was performed...
20 CitationsSource
#1Bertram J. JobstH-Index: 11
#2Oliver Weinheimer (Heidelberg University)H-Index: 8
Last. Mark O. WielpützH-Index: 23
view all 12 authors...
Objective To longitudinally evaluate effects of smoking cessation on quantitative CT in a lung cancer screening cohort of heavy smokers over 4 years.
12 CitationsSource
#1Daiwei Han (UMCG: University Medical Center Groningen)H-Index: 3
#2Marjolein A Heuvelmans (Medisch Spectrum Twente)H-Index: 5
Last. Matthijs OudkerkH-Index: 94
view all 10 authors...
Objective:To evaluate the influence of nodule margin on inter- and intrareader variability in manual diameter measurements and semi-automatic volume measurements of solid nodules detected in low-dose CT lung cancer screening.Methods:25 nodules of each morphological category (smooth, lobulated, spiculated and irregular) were randomly selected from 93 participants of the Dutch-Belgian Randomized Lung Cancer Screening Trial (NELSON). Semi-automatic volume measurements were performed using Syngo Lun...
16 CitationsSource
#1Henry M. Marshall (UQ: University of Queensland)H-Index: 14
#2Henry Zhao (UQ: University of Queensland)H-Index: 2
Last. Kwun M. Fong (UQ: University of Queensland)H-Index: 72
view all 7 authors...
High false-positive (FP) scan rates associated with low-dose computed tomography (LDCT) lung cancer screening result in unnecessary follow-up tests and exposure to harm. The definition of a ‘positive’ scan can impact FP rates and screening performance. We explored the effect of Lung Imaging Reporting and Data System (Lung-RADS) criteria, PanCan Nodule Malignancy Probability Model and varying nodule size thresholds (≥4 mm, ≥6 mm, ≥8 mm) on diagnostic accuracy and screening performance compared wi...
9 CitationsSource
#1Viswam S. Nair (USF: University of South Florida)H-Index: 17
#2Vandana Sundaram (Stanford University)H-Index: 25
Last. Michael K. Gould (KP: Kaiser Permanente)H-Index: 67
view all 4 authors...
16 CitationsSource
Cited By12
#1Stephen Lam (UBC: University of British Columbia)H-Index: 86
#2Martin C. Tammemagi (Brock University)H-Index: 29
Lung cancer screening with low-dose computed tomography can reduce death from lung cancer by 20-24% in high-risk smokers. National lung cancer screening programmes have been implemented in the USA and Korea and are being implemented in Europe, Canada and other countries. Lung cancer screening is a process, not a test. It requires an organised programmatic approach to replicate the lung cancer mortality reduction and safety of pivotal clinical trials. Cost-effectiveness of a screening programme i...
#1Stefan DelormeH-Index: 63
#2Rudolf KaaksH-Index: 151
Last. Rudolf Kaaks (DKFZ: German Cancer Research Center)H-Index: 12
view all 2 authors...
Purpose For screening with low-dose CT (LDCT) to be effective, the benefits must outweigh the potential risks. In large lung cancer screening studies, a mortality reduction of approx. 20 % has been reported, which requires several organizational elements to be achieved in practice. Materials and methods The elements to be set up are an effective invitation strategy, uniform and quality-assured assessment criteria, and computer-assisted evaluation tools resulting in a nodule management algorithm ...
1 CitationsSource
#2Arnaud Arindra Adiyoso Setio (Siemens)H-Index: 12
Last. Colin JacobsH-Index: 24
view all 10 authors...
Background Accurate estimation of the malignancy risk of pulmonary nodules at chest CT is crucial for optimizing management in lung cancer screening. Purpose To develop and validate a deep learning (DL) algorithm for malignancy risk estimation of pulmonary nodules detected at screening CT. Materials and Methods In this retrospective study, the DL algorithm was developed with 16 077 nodules (1249 malignant) collected -between 2002 and 2004 from the National Lung Screening Trial. External validati...
1 CitationsSource
#1Rafael Paez (VUMC: Vanderbilt University Medical Center)
#2Michael N. KammerH-Index: 5
Last. Pierre P. MassionH-Index: 65
view all 3 authors...
Purpose of review Lung cancer remains the leading cause of cancer-related death in the United States, with poor overall 5-year survival. Early detection and diagnosis are key to survival as demonstrated in lung cancer screening trials. However, with increasing implementation of screening guidelines and use of computed tomography, there has been a sharp rise in the incidence of indeterminate pulmonary nodules (IPNs). Risk stratification of IPNs, particularly those in the intermediate-risk categor...
#1Matthijs OudkerkH-Index: 94
#2Shiyuan Liu ('SMMU': Second Military Medical University)H-Index: 3
Last. John K. Field (University of Liverpool)H-Index: 86
view all 5 authors...
In the past decade, the introduction of molecularly targeted agents and immune-checkpoint inhibitors has led to improved survival outcomes for patients with advanced-stage lung cancer; however, this disease remains the leading cause of cancer-related mortality worldwide. Two large randomized controlled trials of low-dose CT (LDCT)-based lung cancer screening in high-risk populations - the US National Lung Screening Trial (NLST) and NELSON - have provided evidence of a statistically significant m...
8 CitationsSource
#1Maria Tsakok (John Radcliffe Hospital)H-Index: 2
#1Maria Tsakok (John Radcliffe Hospital)H-Index: 7
Last. Fergus Gleeson (John Radcliffe Hospital)H-Index: 6
view all 6 authors...
Abstract Purpose To determine how implementation of an artificial intelligence nodule algorithm, the Lung Cancer Prediction Convolutional Neural Network (LCP-CNN), at the point of incidental nodule detection would have influenced further investigation and management using a series of threshold scores at both the benign and malignant end of the spectrum. Method An observational retrospective study was performed in the assessment of nodules between 5-15 mm (158 benign, 32 malignant) detected on CT...
#2Theron Johnson (DKFZ: German Cancer Research Center)H-Index: 20
Last. Rudolf Kaaks (DKFZ: German Cancer Research Center)H-Index: 12
view all 5 authors...
Background Tumor-associated autoantibodies are considered promising markers for early lung cancer detection; so far, however, their capacity to detect cancer has been tested mostly in a clinical context, but not in population screening settings. This study evaluates the early detection accuracy, in terms of sensitivity and specificity, of EarlyCDT®-Lung-a test panel of seven tumor-associated autoantibodies optimized for lung cancer detection-using blood samples originally collected as part of th...
2 CitationsSource
#1D. BlancH-Index: 1
#2V. RacineH-Index: 1
Last. J. BrehantH-Index: 1
view all 30 authors...
Abstract Purpose The purpose of this study was to create an algorithm to detect and classify pulmonary nodules in two categories based on their volume greater than 100 mm3 or not, using machine learning and deep learning techniques. Materials and method The dataset used to train the model was provided by the organization team of the SFR (French Radiological Society) Data Challenge 2019. An asynchronous and parallel 3-stages pipeline was developed to process all the data (a data “pre-processing” ...
7 CitationsSource
#1Nicole Rankin (USYD: University of Sydney)H-Index: 18
#2Annette McWilliams (UWA: University of Western Australia)H-Index: 35
Last. Henry M. Marshall (UQ: University of Queensland)H-Index: 14
view all 3 authors...
Lung cancer is the number one cause of cancer death worldwide. The benefits of lung cancer screening to reduce mortality and detect early-stage disease are no longer in any doubt based on the results of two landmark trials using LDCT. Lung cancer screening has been implemented in the US and South Korea and is under consideration by other communities. Successful translation of demonstrated research outcomes into the routine clinical setting requires careful implementation and co-ordinated input f...
2 CitationsSource
#1Stephen Lam (UBC: University of British Columbia)H-Index: 86
#2Heather BryantH-Index: 26
Last. Kazuhiro YasufukuH-Index: 61
view all 27 authors...
RATIONALE: Appropriate management of low-dose computed tomography (LDCT) screening detected lung nodules will have significant implications for health care resource utilization and minimizing harm ...
2 CitationsSource