Segmentation of organs-at-risks in head and neck CT images using convolutional neural networks

Published on Feb 1, 2017in Medical Physics3.317
· DOI :10.1002/MP.12045
Bulat Ibragimov19
Estimated H-index: 19
(Stanford University),
Lei Xing85
Estimated H-index: 85
(Stanford University)
Purpose Accurate segmentation of organs-at-risks (OARs) is the key step for efficient planning of radiation therapy for head and neck (HaN) cancer treatment. In the work, we proposed the first deep learning-based algorithm, for segmentation of OARs in HaN CT images, and compared its performance against state-of-the-art automated segmentation algorithms, commercial software, and interobserver variability. Methods Convolutional neural networks (CNNs)—a concept from the field of deep learning—were used to study consistent intensity patterns of OARs from training CT images and to segment the OAR in a previously unseen test CT image. For CNN training, we extracted a representative number of positive intensity patches around voxels that belong to the OAR of interest in training CT images, and negative intensity patches around voxels that belong to the surrounding structures. These patches then passed through a sequence of CNN layers that captured local image features such as corners, end-points, and edges, and combined them into more complex high-order features that can efficiently describe the OAR. The trained network was applied to classify voxels in a region of interest in the test image where the corresponding OAR is expected to be located. We then smoothed the obtained classification results by using Markov random fields algorithm. We finally extracted the largest connected component of the smoothed voxels classified as the OAR by CNN, performed dilate–erode operations to remove cavities of the component, which resulted in segmentation of the OAR in the test image. Results The performance of CNNs was validated on segmentation of spinal cord, mandible, parotid glands, submandibular glands, larynx, pharynx, eye globes, optic nerves, and optic chiasm using 50 CT images. The obtained segmentation results varied from 37.4% Dice coefficient (DSC) for chiasm to 89.5% DSC for mandible. We also analyzed the performance of state-of-the-art algorithms and commercial software reported in the literature, and observed that CNNs demonstrate similar or superior performance on segmentation of spinal cord, mandible, parotid glands, larynx, pharynx, eye globes, and optic nerves, but inferior performance on segmentation of submandibular glands and optic chiasm. Conclusion We concluded that convolution neural networks can accurately segment most of OARs using a representative database of 50 HaN CT images. At the same time, inclusion of additional information, for example, MR images, may be beneficial to some OARs with poorly visible boundaries.
📖 Papers frequently viewed together
2015MICCAI: Medical Image Computing and Computer-Assisted Intervention
3 Authors (Olaf Ronneberger, ..., Thomas Brox)
23.2k Citations
2,659 Citations
2016CVPR: Computer Vision and Pattern Recognition
4 Authors (Kaiming He, ..., Jian Sun)
68.6k Citations
#1Ching-Wei Wang (NTUST: National Taiwan University of Science and Technology)H-Index: 19
#2Cheng-Ta Huang (NTUST: National Taiwan University of Science and Technology)H-Index: 10
Last. Claudia Lindner (University of Manchester)H-Index: 12
view all 13 authors...
Abstract Dental radiography plays an important role in clinical diagnosis, treatment and surgery. In recent years, efforts have been made on developing computerized dental X-ray image analysis systems for clinical usages. A novel framework for objective evaluation of automatic dental radiography analysis algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2015 Bitewing Radiography Caries Detection Challenge and Cephalometric X-ray Image An...
119 CitationsSource
#1Jianhua Yao (NIH: National Institutes of Health)H-Index: 40
#2Joseph E. Burns (UCI: University of California, Irvine)H-Index: 15
Last. Shuo Li (UWO: University of Western Ontario)H-Index: 39
view all 16 authors...
A multiple center milestone study of clinical vertebra segmentation is presented in this paper. Vertebra segmentation is a fundamental step for spinal image analysis and intervention. The first half of the study was conducted in the spine segmentation challenge in 2014 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) Workshop on Computational Spine Imaging (CSI 2014). The objective was to evaluate the performance of several state-of-the-art vertebra...
73 CitationsSource
#2Gemma EminowiczH-Index: 1
Last. Sebastien OurselinH-Index: 91
view all 12 authors...
Purpose: The aim of this study was to assess whether clinically acceptable segmentations of organs at risk (OARs) in head and neck cancer can be obtained automatically and efficiently using the novel “similarity and truth estimation for propagated segmentations” (STEPS) compared to the traditional “simultaneous truth and performance level estimation” (STAPLE) algorithm. Methods: First, 6 OARs were contoured by 2 radiation oncologists in a dataset of 100 patients with head and neck cancer on plan...
40 CitationsSource
#1Valerio Fortunati (EUR: Erasmus University Rotterdam)H-Index: 9
#2René F. Verhaart (EUR: Erasmus University Rotterdam)H-Index: 12
Last. Theo van Walsum (EUR: Erasmus University Rotterdam)H-Index: 27
view all 6 authors...
A hyperthermia treatment requires accurate, patient-specific treatment planning. This planning is based on 3D anatomical models which are generally derived from computed tomography. Because of its superior soft tissue contrast, magnetic resonance imaging (MRI) information can be introduced to improve the quality of these 3D patient models and therefore the treatment planning itself. Thus, we present here an automatic atlas-based segmentation algorithm for MR images of the head and neck. Our meth...
16 CitationsSource
#1Christian Wachinger (MIT: Massachusetts Institute of Technology)H-Index: 25
#2Karl D. Fritscher (Harvard University)H-Index: 16
Last. Polina Golland (MIT: Massachusetts Institute of Technology)H-Index: 52
view all 4 authors...
We propose new methods for automatic segmentation of images based on an atlas of manually labeled scans and contours in the image. First, we introduce a Bayesian framework for creating initial label maps from manually annotated training images. Within this framework, we model various registration- and patch-based segmentation techniques by changing the deformation field prior. Second, we perform contour-driven regression on the created label maps to refine the segmentation. Image contours and im...
32 CitationsSource
#1Chang Juan Tao (SYSU: Sun Yat-sen University)H-Index: 6
#2Jun lin Yi (Peking Union Medical College)H-Index: 3
Last. Ying Sun (SYSU: Sun Yat-sen University)H-Index: 54
view all 16 authors...
Abstract Background and purpose To assess whether consensus guideline-based atlas-based auto-segmentation (ABAS) reduces interobserver variation and improves dosimetric parameter consistency for organs at risk (OARs) in nasopharyngeal carcinoma (NPC). Materials and methods Eight radiation oncologists from 8 institutes contoured 20 OARs on planning CT images of 16 patients via manual contouring and manually-edited ABAS contouring. Interobserver variation [volume coefficient of variation (CV), Dic...
48 CitationsSource
#1Yann LeCun (NYU: New York University)H-Index: 119
#2Yoshua Bengio (UdeM: Université de Montréal)H-Index: 192
Last. Geoffrey E. Hinton (U of T: University of Toronto)H-Index: 157
view all 3 authors...
Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its inter...
30.7k CitationsSource
This report provides an overview of the current state of the art deep learning architectures and optimisation techniques, and uses the ADNI hippocampus MRI dataset as an example to compare the effectiveness and efficiency of different convolutional architectures on the task of patch-based 3-dimensional hippocampal segmentation, which is important in the diagnosis of Alzheimer's Disease. We found that a slightly unconventional "stacked 2D" approach provides much better classification performance ...
49 Citations
#1Ching-Wei Wang (NTUST: National Taiwan University of Science and Technology)H-Index: 19
#2Cheng-Ta Huang (NTUST: National Taiwan University of Science and Technology)H-Index: 10
Last. Bulat Ibragimov (University of Ljubljana)H-Index: 19
view all 17 authors...
Cephalometric analysis is an essential clinical and research tool in orthodontics for the orthodontic analysis and treatment planning. This paper presents the evaluation of the methods submitted to the Automatic Cephalometric X-Ray Landmark Detection Challenge, held at the IEEE International Symposium on Biomedical Imaging 2014 with an on-site competition. The challenge was set to explore and compare automatic landmark detection methods in application to cephalometric X-ray images. Methods were ...
66 CitationsSource
#1Lindsey A. Torre (ACS: American Cancer Society)H-Index: 16
#2Freddie Bray (IARC: International Agency for Research on Cancer)H-Index: 119
Last. Ahmedin Jemal (ACS: American Cancer Society)H-Index: 139
view all 6 authors...
Cancer constitutes an enormous burden on society in more and less economically developed countries alike. The occurrence of cancer is increasing because of the growth and aging of the population, as well as an increasing prevalence of established risk factors such as smoking, overweight, physical inactivity, and changing reproductive patterns associated with urbanization and economic development. Based on GLOBOCAN estimates, about 14.1 million new cancer cases and 8.2 million deaths occurred in ...
18.1k CitationsSource
Cited By255
#1Xianjin Dai (Emory University)H-Index: 9
#2Yang Lei (Emory University)H-Index: 22
Last. Xiaofeng Yang (Emory University)H-Index: 28
view all 10 authors...
Purpose null Auto-segmentation algorithms offer a potential solution to eliminate the labor-intensive, time-consuming, and observer-dependent manual delineation of organs-at-risk (OARs) in radiotherapy treatment planning. This study aimed to develop a deep learning-based automated OAR delineation method to tackle the current challenges remaining in achieving reliable expert performance with the state-of-the-art auto-delineation algorithms. null Methods null The accuracy of OAR delineation is exp...
#1Gihan Samarasinghe (UNSW: University of New South Wales)H-Index: 3
#2Michael G Jameson (UNSW: University of New South Wales)H-Index: 9
Last. Lois C Holloway (Liverpool Hospital)H-Index: 20
view all 7 authors...
Segmentation of organs and structures, as either targets or organs-at-risk, has a significant influence on the success of radiation therapy. Manual segmentation is a tedious and time-consuming task for clinicians, and inter-observer variability can affect the outcomes of radiation therapy. The recent hype over deep neural networks has added many powerful auto-segmentation methods as variations of convolutional neural networks (CNN). This paper presents a descriptive review of the literature on d...
#1Jiangchang Xu (SJTU: Shanghai Jiao Tong University)H-Index: 1
#2Jiannan Liu (SJTU: Shanghai Jiao Tong University)H-Index: 1
Last. Xiaojun Chen (SJTU: Shanghai Jiao Tong University)H-Index: 17
view all 7 authors...
In cranio-maxillofacial surgery, it is of great clinical significance to segment mandible accurately and automatically from CT images. However, the connected region and blurred boundary in teeth and condyles make the process challenging. At present, the mandible is commonly segmented by experienced doctors using manually or semi-automatic methods, which is time-consuming and has poor segmentation consistency. In addition, existing automatic segmentation methods still have problems such as region...
Background: Over half a million individuals are diagnosed with head and neck cancer each year globally. Radiotherapy is an important curative treatment for this disease, but it requires manual time to delineate radiosensitive organs at risk. This planning process can delay treatment while also introducing interoperator variability, resulting in downstream radiation dose differences. Although auto-segmentation algorithms offer a potentially time-saving solution, the challenges in defining, quanti...
#1Zhuangzhuang ZhangH-Index: 2
#2Tianyu ZhaoH-Index: 1
Last. Baozhou SunH-Index: 14
view all 4 authors...
In radiotherapy planning, manual contouring is labor-intensive and time-consuming. Accurate and robust automated segmentation models improve the efficiency and treatment outcome. We aim to develop a novel hybrid deep learning approach, combining convolutional neural networks (CNNs) and the self-attention mechanism, for rapid and accurate multi-organ segmentation on head and neck computed tomography (CT) images. Head and neck CT images with manual contours of 115 patients were retrospectively col...
#1Matthew Field (UNSW: University of New South Wales)H-Index: 10
#2Nicholas Hardcastle (Peter MacCallum Cancer Centre)H-Index: 21
Last. Lois C HollowayH-Index: 20
view all 5 authors...
Abstract null null Machine learning technology has a growing impact on radiation oncology with an increasing presence in research and industry. The prevalence of diverse data including 3D imaging and the 3D radiation dose delivery presents potential for future automation and scope for treatment improvements for cancer patients. Harnessing this potential requires standardization of tools and data, and focused collaboration between fields of expertise. The rapid advancement of radiation oncology t...
#1Jialin Shi (THU: Tsinghua University)
#1Jialin Shi (THU: Tsinghua University)H-Index: 1
Last. Ji Wu (THU: Tsinghua University)H-Index: 20
view all 6 authors...
PURPOSE Accurately delineating clinical target volumes (CTV) is essential for completing radiotherapy plans but is time-consuming, labor-intensive and prone to inter-observer variation. Automating CTV delineation has the benefits of both speeding up contouring process and improving the quality of contours. Recently, auto-segmentation approaches based on deep learning have achieved some improvements. However, unlike organ segmentation, the CTV contains potential tumor spread tissues or subclinica...
1 CitationsSource
#1Jared Hamwood (QUT: Queensland University of Technology)H-Index: 4
#2Beat Schmutz (QUT: Queensland University of Technology)H-Index: 14
Last. David Alonso-Caneiro (QUT: Queensland University of Technology)H-Index: 24
view all 5 authors...
This paper proposes a fully automatic method to segment the inner boundary of the bony orbit in two different image modalities: magnetic resonance imaging (MRI) and computed tomography (CT). The method, based on a deep learning architecture, uses two fully convolutional neural networks in series followed by a graph-search method to generate a boundary for the orbit. When compared to human performance for segmentation of both CT and MRI data, the proposed method achieves high Dice coefficients on...
This study aims to develop a deep learning-based strategy for treatment plan check and verification of high-dose rate (HDR) brachytherapy. A deep neural network was trained to verify the dwell positions and times for a given input brachytherapy isodose distribution. In our modeling, each dwell position is represented by a Gaussian heatmap located in the vicinity of the dwell positions. A deep inception network based architecture was established to learn the mapping between CT, dose distribution ...