DeepGestalt - Identifying Rare Genetic Syndromes Using Deep Learning

Yaron Gurovich6
Estimated H-index: 6
Yair Hanani4
Estimated H-index: 4
+ 8 AuthorsKaren W. Gripp36
Estimated H-index: 36
Facial analysis technologies have recently measured up to the capabilities of expert clinicians in syndrome identification. To date, these technologies could only identify phenotypes of a few diseases, limiting their role in clinical settings where hundreds of diagnoses must be considered. We developed a facial analysis framework, DeepGestalt, using computer vision and deep learning algorithms, that quantifies similarities to hundreds of genetic syndromes based on unconstrained 2D images. DeepGestalt is currently trained with over 26,000 patient cases from a rapidly growing phenotype-genotype database, consisting of tens of thousands of validated clinical cases, curated through a community-driven platform. DeepGestalt currently achieves 91% top-10-accuracy in identifying over 215 different genetic syndromes and has outperformed clinical experts in three separate experiments. We suggest that this form of artificial intelligence is ready to support medical genetics in clinical and laboratory practices and will play a key role in the future of precision medicine.
Figures & Tables
📖 Papers frequently viewed together
23 Citations
7 Citations
11 Authors (Shahad Kudama, ..., Nicholas W. Wood)
#1Jean Tori Pantel (Charité)H-Index: 4
#2Max Zhao (Charité)H-Index: 3
Last. Peter KrawitzH-Index: 31
view all 11 authors...
Significant improvements in automated image analysis have been achieved over the recent years and tools are now increasingly being used in computer-assisted syndromology. However, the recognizability of the facial gestalt might depend on the syndrome and may also be confounded by severity of phenotype, size of available training sets, ethnicity, age, and sex. Therefore, benchmarking and comparing the performance of deep-learned classification processes is inherently difficult. For a systematic a...
3 CitationsSource
#1Smail Hadj-Rabia (Paris V: Paris Descartes University)H-Index: 32
#2Holm SchneiderH-Index: 33
Last. Dorothy K. Grange (WashU: Washington University in St. Louis)H-Index: 36
view all 11 authors...
X-linked hypohidrotic ectodermal dysplasia (XLHED) is a genetic disorder that affects ectodermal structures and presents with a characteristic facial appearance. The ability of automated facial recognition technology to detect the phenotype from images was assessed . In Phase 1 of this study we examined if the age of male patients affected the technology's recognition. In Phase 2 we investigated how well the technology discriminated affected males cases from female carriers and from individuals ...
18 CitationsSource
#1Hui Ding (SJTU: Shanghai Jiao Tong University)H-Index: 8
#2Shaohua Kevin Zhou (Siemens)H-Index: 9
Last. Rama Chellappa (UMD: University of Maryland, College Park)H-Index: 102
view all 3 authors...
Relatively small data sets available for expression recognition research make the training of deep networks very challenging. Although fine-tuning can partially alleviate the issue, the performance is still below acceptable levels as the deep features probably contain redundant information from the pretrained domain. In this paper, we present FaceNet2ExpNet, a novel idea to train an expression recognition network based on static images. We first propose a new distribution function to model the h...
125 CitationsSource
#1Paul Kruszka (NIH: National Institutes of Health)H-Index: 15
#2Yonit A. Addissie (NIH: National Institutes of Health)H-Index: 7
Last. Maximilian Muenke (NIH: National Institutes of Health)H-Index: 74
view all 39 authors...
22q11.2 deletion syndrome (22q11.2 DS) is the most common microdeletion syndrome and is underdiagnosed in diverse populations. This syndrome has a variable phenotype and affects multiple systems, making early recognition imperative. In this study, individuals from diverse populations with 22q11.2 DS were evaluated clinically and by facial analysis technology. Clinical information from 106 individuals and images from 101 were collected from individuals with 22q11.2 DS from 11 countries; average a...
53 CitationsSource
Mar 24, 2017 in WACV (Workshop on Applications of Computer Vision)
#1Pushkar Shukla (IITR: Indian Institute of Technology Roorkee)H-Index: 4
#2Tanu GuptaH-Index: 3
Last. R. Balasubramanian (IITR: Indian Institute of Technology Roorkee)H-Index: 6
view all 5 authors...
Developmental Disorders are chronic disabilities that have a severe impact on the day to day functioning of a large section of the human population. Recognizing developmental disorders from facial images is an important but a relatively unexplored challenge in the field of computer vision. This paper proposes a novel framework to detect developmental disorders from facial images. A spectrum of disorders constituting of Autism Spectrum Disorder, Cerebral Palsy, Fetal Alcohol Syndrome, Down syndro...
17 CitationsSource
#1Paul Kruszka (NIH: National Institutes of Health)H-Index: 15
#2Antonio R. PorrasH-Index: 9
Last. Maximilian Muenke (NIH: National Institutes of Health)H-Index: 74
view all 44 authors...
Down syndrome is the most common cause of cognitive impairment and presents clinically with universally recognizable signs and symptoms. In this study, we focus on exam findings and digital facial analysis technology in individuals with Down syndrome in diverse populations. Photos and clinical information were collected on 65 individuals from 13 countries, 56.9% were male and the average age was 6.6 years (range 1 month to 26 years; SD = 6.6 years). Subjective findings showed that clinical featu...
41 CitationsSource
#1Karen W. Gripp (DuPont)H-Index: 15
#2Laura Baker (DuPont)H-Index: 5
Last. Kristin G. Monaghan (GeneDx)H-Index: 21
view all 4 authors...
The genetic basis of numerous intellectual disability (ID) syndromes has recently been identified by applying exome analysis on a research or clinical basis. There is significant clinical overlap of biologically related syndromes, as exemplified by Nicolaides-Baraitser (NCBRS) and Coffin-Siris (CSS) syndrome. Both result from mutations affecting the BAF (mSWI/SNF) complex and belong to the growing category of BAFopathies. In addition to the notable clinical overlap between these BAFopathies, het...
24 CitationsSource
#2Lior Wolf (TAU: Tel Aviv University)H-Index: 52
Last. Matthew A. Deardoff (UPenn: University of Pennsylvania)H-Index: 1
view all 7 authors...
Facial analysis systems are becoming available to healthcare providers to aid in the recognition of dysmorphic phenotypes associated with a multitude of genetic syndromes. These technologies automatically detect facial points and extract various measurements from images to recognize dysmorphic features and evaluate similarities to known facial patterns (gestalts). To evaluate such systems' usefulness for supporting the clinical practice of healthcare professionals, the recognition accuracy of th...
46 CitationsSource
#1Karim Kouz (UHH: University of Hamburg)H-Index: 6
#2Christina LissewskiH-Index: 11
Last. Martin ZenkerH-Index: 54
view all 26 authors...
Purpose: Noonan syndrome (NS) is an autosomal-dominant disorder characterized by craniofacial dysmorphism, growth retardation, cardiac abnormalities, and learning difficulties. It belongs to the RASopathies, which are caused by germ-line mutations in genes encoding components of the RAS mitogen-activated protein kinase (MAPK) pathway. RIT1 was recently reported as a disease gene for NS, but the number of published cases is still limited. Methods: We sequenced RIT1 in 310 mutation-negative indivi...
38 CitationsSource
#1Shan Suthaharan (UNCG: University of North Carolina at Greensboro)H-Index: 15
Support Vector Machine is one of the classical machine learning techniques that can still help solve big data classification problems. Especially, it can help the multidomain applications in a big data environment. However, the support vector machine is mathematically complex and computationally expensive. The main objective of this chapter is to simplify this approach using process diagrams and data flow diagrams to help readers understand theory and implement it successfully. To achieve this o...
759 CitationsSource
Cited By16
#1Katharina Schultebraucks (Columbia University)H-Index: 9
#2Bernard P. Chang (Columbia University)H-Index: 16
Abstract Personalized medicine is among the most exciting innovations in recent clinical research, offering the opportunity for tailored screening and management at the individual level. Biomarker-enriched clinical trials have shown increased efficiency and informativeness in cancer research due to the selective exclusion of patients unlikely to benefit. In acute stress situations, clinically significant decisions are often made in time-sensitive manners and providers may be pressed to make deci...
#1Kimon Kieslich (HHU: University of Düsseldorf)H-Index: 1
#2Marco Lünich (HHU: University of Düsseldorf)H-Index: 3
Last. Frank Marcinkowski (HHU: University of Düsseldorf)H-Index: 4
view all 3 authors...
In recent years Artificial Intelligence (AI) has gained much popularity, with the scientific community as well as with the public. Often, AI is ascribed many positive impacts for different social domains such as medicine and the economy. On the other side, there is also growing concern about its precarious impact on society and individuals, respectively. Several opinion polls frequently query the public fear of autonomous robots and artificial intelligence, a phenomenon coming also into scholarl...
#1Kimon KieslichH-Index: 1
#2Marco LünichH-Index: 3
Last. Frank MarcinkowskiH-Index: 4
view all 3 authors...
In recent years Artificial Intelligence (AI) has gained much popularity, with the scientific community as well as with the public. AI is often ascribed many positive impacts for different social domains such as medicine and the economy. On the other side, there is also growing concern about its precarious impact on society and individuals. Several opinion polls frequently query the public fear of autonomous robots and artificial intelligence (FARAI), a phenomenon coming also into scholarly focus...
1 Citations
#1Martin G. Seneviratne (Stanford University)H-Index: 8
#2Nigam H. Shah (Stanford University)H-Index: 53
Last. Larry F. Chu (Stanford University)H-Index: 27
view all 3 authors...
Applications of machine learning on clinical data are now attaining levels of performance that match or exceed human clinicians.1–3 Fields involving image interpretation—radiology, pathology and dermatology—have led the charge due to the power of convolutional neural networks, the existence of standard data formats and large data repositories. We have also seen powerful diagnostic and predictive algorithms built using a range of other data, including electronic health records (EHR), -omics, moni...
This work describes an automatic methodology to discriminate between individuals with the genetic disorder Pitt-Hopkins syndrome (PTHS), and healthy individuals. As input data, the methodology accepts unconstrained frontal facial photographs, from which faces are located with Histograms of Oriented Gradients features descriptors. Pre-processing steps of the methodology consist of colour normalisation, scaling down, rotation, and cropping in order to produce a series of images of faces with consi...
#1Lynnea Myers (KI: Karolinska Institutet)H-Index: 4
#2Britt-Marie Anderlid (KI: Karolinska Institutet)H-Index: 31
Last. Sven Bölte (KI: Karolinska Institutet)H-Index: 51
view all 7 authors...
Physical examinations are recommended as part of a comprehensive evaluation for individuals with neurodevelopmental disorders (NDDs), such as autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder. These examinations should include assessment for morphological variants. Previous studies have shown an increase in morphological variants in individuals with NDDs, particularly ASD, and that these variants may be present in greater amounts in individuals with genetic alterations....
2 CitationsSource
: The diagnosis of rare genetic diseases is one of the most difficult areas in medicine. Whole-exome sequencing (WES) technology makes it easier to diagnose these diseases. In addition, next-generation phenotyping can help to diagnose computer-based algorithms. Detailed dysmorphologic findings of 25 patients diagnosed by WES in our center were described. The success of this technology in diagnosing rare genetic diseases was investigated by scanning the photographs of 25 patients with Face2Gene a...
5 CitationsSource
Genetic diseases are the most common next-generation diseases because of the improper mutation of the genes and DNA. These genetic diseases are failed to predict with an accurate manner in the beginning stage by using the particular genes and related information. So, the genetic diseases are identified in the medical systems by utilizing the hybridization of multimedia techniques such as big data and related soft computing techniques.Initially, the genetic disease-related medical images are coll...
#1Tzung-Chien Hsieh (Humboldt University of Berlin)H-Index: 5
#2Martin Atta Mensah (Humboldt University of Berlin)H-Index: 4
Last. Peter Krawitz (University of Bonn)H-Index: 31
view all 94 authors...
Phenotype information is crucial for the interpretation of genomic variants. So far it has only been accessible for bioinformatics workflows after encoding into clinical terms by expert dysmorphologists. Here, we introduce an approach driven by artificial intelligence that uses portrait photographs for the interpretation of clinical exome data. We measured the value added by computer-assisted image analysis to the diagnostic yield on a cohort consisting of 679 individuals with 105 different mono...
13 CitationsSource