Clinical Ophthalmology Dovepress open access to scientific and medical research Open Access Full Text Article ORIGINAL RESEARCH Artificial Intelligence to Identify Retinal Fundus Images, Quality Validation, Laterality Evaluation, Macular Degeneration, and Suspected Glaucoma This article was published in the following Dove Press journal: Clinical Ophthalmology Miguel Angel Zapata 1 Purpose: To assess the performance of deep learning algorithms for different tasks in retinal Dídac Royo-Fibla 1 fundus images: (1) detection of retinal fundus images versus optical coherence tomography (OCT) Octavi Font 1 or other images, (2) evaluation of good quality retinal fundus images, (3) distinction between right José Ignacio Vela 2,3 eye (OD) and left eye (OS) retinal fundus images,(4) detection of age-related macular degeneration (AMD) and (5) detection of referable glaucomatous optic neuropathy (GON). Ivanna Marcantonio 2,3 Patients and Methods: Five algorithms were designed. Retrospective study from a database Eduardo Ulises Moya- of 306,302 images, Optretina’s tagged dataset. Three different ophthalmologists, all retinal Sánchez 4,5 specialists, classified all images. The dataset was split per patient in a training (80%) and testing Abraham Sánchez-Pérez 5 (20%) splits. Three different CNN architectures were employed, two of which were custom Darío Garcia-Gasulla 4 designed to minimize the number of parameters with minimal impact on its accuracy. Main Ulises Cortés 4,6 outcome measure was area under the curve (AUC) with accuracy, sensitivity and specificity. Eduard Ayguadé 4,6 Results: Determination of retinal fundus image had AUC of 0.979 with an accuracy of 96% Jesus Labarta 4,6 (sensitivity 97.7%, specificity 92.4%). Determination of good quality retinal fundus image 1 Optretina, Barcelona, Spain; had AUC of 0.947, accuracy 91.8% (sensitivity 96.9%, specificity 81.8%). Algorithm for 2 Ophthalmology Department, Hospital OD/OS had AUC 0.989, accuracy 97.4%. AMD had AUC of 0.936, accuracy 86.3% de la Santa Creu I de Sant Pau, Barcelona (sensitivity 90.2% specificity 82.5%), GON had AUC of 0.863, accuracy 80.2% (sensitivity 08041, Spain; 3Universitat Autònoma de Barcelona (UAB), Campus de la UAB, 76.8%, specificity 83.8%). Barcelona, Spain; 4Barcelona Conclusion: Deep learning algorithms can differentiate a retinal fundus image from other Supercomputing Center (BSC), images. Algorithms can evaluate the quality of an image, discriminate between right or left Barcelona, Spain; 5Universidad Autónoma de Guadalajara - Postgrado en Ciencias eye and detect the presence of AMD and GON with a high level of accuracy, sensitivity and Computacionales, Guadalajara, Mexico; specificity. 6 Universitat Politècnica de Catalunya - BarcelonaTECH, Campus Nord, Keywords: artificial intelligence, retinal diseases, screening, retinal fundus image Barcelona, Spain Introduction Retinal diseases are on the rise due to the increase of diabetic population and increased longevity. A good example of the effect of age on retinal health is age-related macular degeneration (AMD): in 2020, the number of people with this disease is projected to be 196 million and is expected to grow to 288 million by 2040.1 Glaucoma is a progressive optic neuropathy, which is the leading cause of blindness in industrialized countries and ocular hypertension is the main risk factor for glaucoma.2 General screening for retinal diseases and glaucoma in the population is ideal, as Correspondence: Miguel Angel Zapata Optretina, C/ Las Palmas 11, 08195 Sant many pathologies can be detected in the early stages, while other diseases—such as Cugat del Vallès, Barcelona, Spain retinal dystrophies, choroidal nevi or epiretinal membranes (ERMs)—can also be Tel +34 655809682 Email mazapata@optretina.com diagnosed and are better managed with early detection. Screening for diabetic submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 419–429 419 DovePress © 2020 Zapata et al. This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms. http://doi.org/10.2147/OPTH.S235751 php and incorporate the Creative Commons Attribution – Non Commercial (unported, v3.0) License (http://creativecommons.org/licenses/by-nc/3.0/). By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms (https://www.dovepress.com/terms.php). Zapata et al Dovepress retinopathy (DR) and AMD is already cost-effective, but Optretina is a telemedicine platform which performs there are no published studies to support the case for general screening for retinal diseases using nonmydriatic general screening yet. Retinal fundus images using non- cameras and human evaluation by a retinal specialist mydriatic cameras (NMC) are currently considered the ophthalmologist.21 Our research in artificial intelligence gold standard for screening DR3,4 and are also cost- deals with both data acquisition and the diagnostic steps effective for AMD.5,6 Similarly, screening for premature of our telemedicine platform, in a near future we expect AI retinopathy is supported by many papers, showing good could assist our specialists in achieving levels of consis- efficacy and cost-effectiveness.7 The detection of disease tency and accuracy beyond unassisted human abilities. through retinal fundus images has also been shown to This study aims to evaluate the capacity of AI methods produce competitive results, particularly for neurological8 for solving several tasks related to our CADx pipeline. We or cardiovascular9 diseases. However, general screenings automatically discard inputs that are not derived from using retinal fundus images have, to the best of our knowl- color fundus photography (CFP); next we filter out images edge, never been implemented—even on particularly sen- that do not pass our quality threshold; we then differentiate sitive segments of the population, such as those over fifty the right eye from the left eye; and finally, we proceed to years old, an age at which retinal pathologies increase the diagnosis, using the remaining images and our algo- significantly. rithms for AMD and suspected glaucoma. Artificial intelligence (AI) and, more specifically, deep learning, could be helpful in this area. Deep learning is a set of machine learning techniques that have recently Methods undergone a renaissance, achieving breakthrough accuracy Datasets in image recognition tasks in different fields from compu- Optretina has been carrying out a telemedicine screening ter vision10 to the medical sciences, solving problems such program since 2013 in optical centers and since 2017 in as: detecting melanoma,11 detecting cancer metastases workplace offices and private companies. This study is from pathology images,12 and diagnosing pneumonia based on the tagged dataset from Optretina, composed of using x-rays.13 We argue that a computer-aided diagnosis 306,302 retinal images. Labelled with diagnosis, laterality system (CADx), whether fully automated or used in con- and quality. Ophthalmologists—all of whom are retinal junction with medical specialists, could substantially specialists—have evaluated all of these images. The data- reduce the human and economic costs of image analysis. set is real-life (Not from clinical trials) and anonymized, Furthermore, in the last few years, NMC have become composed of approximately 80% cases of normal retinas significantly cheaper; in the near future, any cell phone and 20% abnormal ones. Images are acquired using NMC could be used as a retinal camera for early screening. (color fundus and red free) and optical coherence tomo- Deep learning has already had success in ophthalmol- graphy (OCT). The dataset includes images from different ogy, demonstrating its ability to detect DR in retinal fun- types and brands of cameras. Figure 1 shows the distribu- dus images.14–17 In fact, previous studies report accuracy tion of the different cameras used, by percentage. The of over 90% with sensitivity and specificity levels over dataset includes color fundus photography (CFP), macu- 90% as well, which coincides with roughly the same level lar OCT’s, retinal nerve fiber layer OCT (RNFL), and as a human ophthalmology expert. We can find similar other added non-medical images trying to fool the AI outcomes with the detection of referable AMD.18 (such as round objects like pizzas or photos of the full However, using such algorithms in real-world situations moon). Of the 306,302 images, over 250,000 correspond remains a challenge, as shown by the decreased accuracy to CFP. In 2017, Optretina published the results of three (around 80%) achieved in prospective studies.19 This years of screenings,17 detailing the percentages of dis- decrease could be due to inconsistent image quality and eases found. other aspects, such as comorbidity. Dataset quality assess- Classification of image type (CFP, OCT, others) and ment is normally taken for granted, with researchers using eye of evaluation (right or left) was performed by one trainings and certifications for photographers17,18 or manu- ophthalmologist. Labeling for quality assessment, pre- ally “cleaning” the datasets;19 some authors have even sence of AMD, or glaucomatous optic neuropathy (GON) used quality filters on algorithms before training the was performed by two independent retinal specialists, algorithms.20 a third ophthalmologist assessed cases where there was submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 420 DovePress Dovepress Zapata et al Figure 1 Distribution of nonmydriatic cameras (NMCs) and optical coherence tomography (OCT) used for the dataset. Only central retinal images were included. no agreement, in order to establish at least two consistent number of images used for each deep learning model, the gradings. Since Optretina performs screenings and patients number of classes per model, and the number of ophthal- in whom pathology is detected are subsequently referred to mologists who independently labeled the same dataset. an ophthalmologist, repetition of images of the same patient are very rare. It is estimated that 0.5% of the Convolutional Neuronal Networks (CNNs) dataset images may be of patients already assessed in The first three CNN models (trained from scratch) were previous years. The dataset was divided by patient in two used for different image classification tasks usually splits: training (80%) and testing (20%). 10% of the train- applied in data-collection, data-selection, display, and ing split was used for validation purposes during the data classification. The first model is very useful for training process of the models. Table 1 displays the cleaning unlabeled datasets, since it can separate the Table 1 Number of Images Used for Training the CNNs. Images Were Divided into Two Groups: One for Training the Model and Another Naïve Group for Validating the Training Procedure. Color Fundus photography (CFP), Optical coherence tomography (OCT), Retinal nerve fiber layer (RNFL), Right eye (OD), Left eye (OS), Age related macular degeneration (AMD), referable glaucomatous optic neuropathy (GON). Model Classification Task Images for Data Augmentation Number of Number of Ophthalmologists for Training from Keras Classes Label the Images 1 Image type selection 56,396 Shear_range=0.2, 4 1 CFP/OCT/RFNL/OTHER zoom_range=0.2 2 Data Collection Good 150,075 Shear_range=0.2, 2 3 quality images zoom_range=0.2 3 Data Display 30,119 Shear_range=0.2, 2 1 OD/OS zoom_range=0.2 4 AMD 8832 Zoom_range 0.1 2 3 rotation_range 10 horizontal_flip: True 5 GON 3776 Zoom_range 0.1 2 3 rotation_range 15 horizontal_flip: True submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 421 DovePress Zapata et al Dovepress CFP images from OCT or other type of images like red- and three dense layers; as a result, the CNN-1 has fewer free image, autofluorescence, angiographic images or parameters than very deep models (such as Inception, VGG- other. The second model allows us to select only the 19, and RESNET50, among others). Its shallow architecture CFP with a minimum threshold of image quality. The makes it computationally inexpensive, which is beneficial third model helps us to classify right-eye CFP versus for increasing the processing throughput of our CADx pipe- left-eye CFP, which we use in our platform to display the line. CNN-1, along with the rest of the models, have been best pair of retinas per patient. The fourth model classi- implemented using Keras,22 a software library that facilitates fies the AMD CFP versus the normal CFP. Finally, the the creation of neural networks with access to specialized fifth model allows for the classification of normal CFP hardware resources, such as graphics processing units versus GON. Table 2 presents the three main CNN. We (GPUs), in order to speed up the training and execution of have also published sample code for these architectures such algorithms. on https://github.com/octavifs/optretina-cnns. Model 2: Good Quality Retinal Fundus Image Model 1: Image Type Selection CFP/OCT/Other Classification Images The second model also implements CNN-1 architec- The CNN-1 performs a binary classification, separating CFP ture, but for a different task: to select good quality from all other image types (OCT/RNFL/other images). The CFP. The definition of “good quality” was defined by training process was done from scratch, using RGB images considering the color retinal fundus image, centered on resized to 128x128 pixels with geometric data augmentation macula with a correct focus; good visualization of the over the training dataset. It should be noted that the CNN-1 parafoveal vessels; and if it was possible to visualize has only three convolution layers, three max pooling layers, over two disc diameters around the fovea, the optic Table 2 CNN Architecture for Algorithms: 1. Color Fundus Photography (CFP) versus Macular Optical Coherence Tomography (OCT), Retinal Nerve Fiber Layer in the OCT and Other images. 2. Good Quality Retinal Fundus image. 3. Right Eye versus Left Eye (OD/OS). 4. AMD. 5. Glaucomatous Optic Neuropathy (GON) Classification. (Notations are Based on Keras22) CNN-1 AMD-Net GON-Net Input (128,128,3) Input (512,512,3) Input (224,224,3) Conv_2D_1(ReLu,3,3,32) Conv_2D_1(ReLu,5,5,32) RESNET50 Max_Pooling_1(2,2) Max_Pooling_1(2,2) Flatten Conv_2D_2(ReLu,3,3,32) Conv_2D_2(ReLu,3,3,32) Dropout_1(0.5) Max_Pooling_2(2,2) Conv_2D_3(ReLu,3,3,32) Dense_1(ReLu,64) Conv_2D_3(ReLu,3,3,64) Max_Pooling_2(2,2) Dropout_2(0.2) Max_Pooling_3(2,2) Conv_2D_4(ReLu,3,3,32) Dense_2 (Softmax, 2) Flatten Conv_2D_5(ReLu,3,3,32) Dense_1 (ReLu,64) Max_Pooling_3(2,2) Dropout_1 (0,5) Conv_2D_6(ReLu,3,3,32) Dense_2 (Sigmoid, 5*/2) Conv_2D_7(ReLu,3,3,32) Max_Pooling_4(2,2) We trained only for 50 epochs, batch Conv_2D_8(ReLu,3,3,32) size of 32 and learning rate of 0.001 for each model. Conv_2D_9(ReLu,3,3,32) Max_Pooling_5(2,2) Conv_2D_10(ReLu,3,3,64) Conv_2D_11(ReLu,3,3,64) Max_Pooling_6(2,2) Flatten Dense_1(ReLu64) Dropout_1(0.5) Dense_2(ReLu,64) Dropout_2(0.2) Dense_2 (Softmax, 2) submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 422 DovePress Dovepress Zapata et al Figure 2 (A) Examples of good quality images. Retinal fundus images are centered on the macula with correct focus, good visualization of the parafoveal vessels and over two disc diameters around the fovea, and correct observation of the optic disc (at least three-quarters) and the vascular arcades. (B) Examples of poor quality images. disc (at least three-quarters), and the vascular arcades. AMD was classified according to the international Figure 2 shows an example of good and bad quality classification published in 2013:23 retinal fundus images. We also trained the same CNN-1 from scratch for a different objective with geometric ● Early AMD: the presence of medium drusen within data augmentation. two disc diameters (DD) of the fovea. ● Intermediate AMD: the presence of large drusen or Model 3: Right-Eye versus Left-Eye (OD/OS) medium drusen with pigmented abnormalities within Classification two DDs from the center. The third model is very useful for automatically displaying ● Advanced AMD: the presence of geographic atrophy both CFP images of the same patient in order for the (> 125 microns) or signs of choroidal neovascular- ophthalmologist to label the images. We trained the ization or fibrosis within two DDs from the fovea. model from scratch with data augmentation and using CNN-1 architecture in order to classify the left eye versus Although the images were classified into the three stages the right eye. It is important to note that the same simple of AMD for further investigations, this first algorithm was architecture was useful for classifying images in different binary, and detected AMD retinas (at any stage) from tasks. However, while the architecture is the same, each normal ones. training process teaches the model the features necessary We propose a novel architecture for AMD binary classi- to learn a specific task. fication which will be referred to hereinafter as AMDNET. The architecture of AMDNET allows us to extract small Model 4: Age-Related Macular Degeneration (AMD) details and features while also making predictions with AMD image classification poses special challenges due to relatively few trainable parameters (around 241,000) in the fact that images are almost identical and only small comparison with other well-known architectures such as features determine their class. RESNET5024 or InceptionV325 which have approximately submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 423 DovePress Zapata et al Dovepress 25 million trainable parameters. The AMDNET architecture characteristics (ROC) curves, area under the curve is presented in Table 2. We trained AMDNET from scratch (AUC), accuracy, and all other calculations were made with the data specifications presented in Table 1. using SPSS 15. The number of images used for each algorithm depended on the number of good images with Algorithm 5: Referable Glaucomatous Optic the disease in the Optretina database. Neuropathy (GON) Referable glaucomatous optic neuropathy (GON) was defined by a cup-to-disc ratio of 0.7 or more in the vertical Results axis and/or other typical changes caused by glaucoma, Table 3 summarizes the outcomes from the different con- such as localized notches or RNFL defects. We modified volutional neuronal networks (CNNs) in the study. Table 4 RESNET50 (23) and training from scratch in order to reflects the confusion matrix in the validation sets, indicat- make a binary GON image classification. ing positives and negatives, comparing human and artifi- cial evaluation. ROC curves are seen in Figure 3. Study of Misclassifications The first CNN differentiating color fundus images from To study the possible cause of misclassification an experi- other medical or non-medical images has an AUC of enced retinal specialist and the Optretina medical director 0.979, with an accuracy of 96%. This algorithm has (Miguel A. Zapata) reviewed all false positives and nega- a sensitivity over 97% and a specificity of 92%. Artificial tives in the AMD and the GON algorithm. intelligence for determining if a retinal fundus image was assessable or not had an accuracy of 92%, compared to Ethics and Data Protection human determination and an AUC of 0.947. This CNN Given that requesting informed consent for the Optretina also had high sensitivity (97%) and specificity (82%). evaluation is mandatory, patients are informed of the pros The algorithm for the right eye/left eye (OD/OS) had and cons of screening, including the spectrum of diseases an AUC of 0.989 with an accuracy of 97.4%. CNNs for that can and cannot be detected by a retinal fundus image. evaluating diseases had an AUC of 0.936 in the case of As part of giving consent, patients also agreed to their age-related macular degeneration (AMD) with an accuracy images being used anonymously for science and research of 86.3%, and an AUC of 0.863 with an 80% accuracy in purposes. The dataset was completely anonymized. No the case of glaucomatous optic neuropathy. The algorithm Optretina or BSC employee has access to patient data for detecting AMD had a sensitivity of 90.2% and except for the Optretina technical director (Didac Royo), a specificity of 82.5%. In the case of glaucomatous optic solely for emergency purposes. The keys for reversing the neuropathy (GON), sensitivity was 76.8% and specificity anonymization are physically stored in a bank. The study was 83.8%. was approved by the ethics committee of the Hospital Vall The first cause of misclassification of images in the d’Hebron in Barcelona, the research followed the tenets of AMD and the GON algorithm was the presence of border- the Declaration of Helsinki. line cases. In the case of AMD, false negatives were early AMD that was not seen by the CNN as disease; false Statistical Analyses positives were cases of small drusen in the macular area A confusion matrix was created from the validation set in that were not tagged as AMD by the retinal specialist. False all five CNNs. Images were divided based on the human negatives and positives in suspected glaucoma were border- and algorithm classification. Receiver operating line cases with approximately 7/10 vertical excavation. Table 3 Outcomes Calculated Using the Confusion Matrix in Different Algorithms Model AUC Accuracy Sensitivity Specificity Image type 0.979 (0.978–0.981) 0.960 (0.957–0.962) 0.977 (0.976–0.979) 0.924 (0.920–0.929) Eye 0.989 (0.988–0.991) 0.974 (0.973–0.977) 0.983 (0.981–0.985) 0.966 (0.963–0.970) Image quality 0.947 (0.945–0.948) 0.918 (0.916–0.919) 0.969 (0.968–0.970) 0.818 (0.814–0.821) AMD classification 0.936 (0.922–0.946) 0.863 (0.845–0.877) 0.902 (0.880–0.916) 0.825 (0.799–0.849) GON classification 0.863 (0.827–0.894) 0.803 (0.760–0.836) 0.768 (0.710–0.824) 0.838 (0.784–0.888) submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 424 DovePress Dovepress Zapata et al Table 4 Validation Confusion Matrix from the Five CNNs: 1. Color uploads the images on the telemedicine platform; in this Fundus Photography (CFP) versus Other Images. 2. Good Quality way, the system can detect that there is at least one retinal Retinal Fundus Images. 3. Right Eye versus Left Eye (OD/OS). 4. Age fundus image for each eye and that the image is of a good Related Macular Degeneration (AMD). 5. Glaucomatous Optic enough quality to be evaluated. Neuropathy (GON) Unlike other publications that aim to replace the human Filetype reading, we will like to explore future models based on the Prediction False True idea that a hybrid reading—or the use of artificial intelli- False 31% 2% gence (AI) as a diagnostic aid tool can, at this time, be True 3% 64% a better solution for both guaranteeing the quality of the Laterality readings and decreasing costs and time invested, while also Prediction False True causing fewer ethical or legal problems. However, when False 48% 1% working with images from different types of cameras, and True 2% 49% even non-retinal fundus image images, studying the image Quality assessment quality is even more important. Prediction False True In the near future, screening for retinal diseases will False 28% 2% most probably be affected by the increasing ease with True 6% 64% which images will be able to be produced due to the AMD decreasing cost of NMCs and the use of smartphones and Prediction False True portable digital devices. As other authors have described, False 42% 5% True 9% 44% these procedures should be autonomous with little require- ment for trained personnel.29 It has been well described that GON the first step necessary for a screening algorithm is the Prediction False True recognition and the exclusion of insufficient image False 42% 11% True 8% 39% quality.29 This is what our first CNNs dealt with. In our case, algorithms for identification and quality assurance have levels of accuracy that are better than human-only. Discussion Retinal landmarks have traditionally been used for recog- For the last three years, artificial intelligence has been consid- nizing a retinal image and evaluating its quality,30,31 but we ered useful for detecting certain pathologies in retinal fundus used two CNNs without any intervention or segmentation, images. Studies show spectacular results for the detection of thereby obtaining high levels of AUC for the algorithms. To guarantee the quality of the images, some authors diabetic retinopathy,14–17 macular degeneration,18,26 and cases perform a manual evaluation of their databases;16,29 while of suspected glaucoma.15,27,28 this system is suitable for retrospective studies, using it in Optretina performs general screening for the main dis- prospective studies is not ideal. Other authors point out the eases that affect the central retina. The screening is based need for training and certifying photographers,19 which on retinal fundus images that use mainly table-top non- certainly substantially improves the image quality. We mydriatic cameras (NMCs), although portable cameras believe, however, that the use of quality algorithms may and optical coherence tomography (OCT) are also used. be a more appropriate measure: it has a short feedback The readings are conducted by specialist retina ophthal- circle, even during the image taking process, thus offering mologists who subsequently issue a report on the state of a greater guarantee of quality and more cost-effectiveness. the patient’s two central retinas. We think, in a near future Other authors have also used quality algorithms, as was artificial intelligence could improve the times and quality the case with Varadarajan et al,20 who had to exclude 12% of reading, resulting in our readers having better outcomes of the images of one of their datasets due to poor quality. than human-only readings. In order for this to occur, we In the case of multiple acquisition devices and different must first guarantee the quality of the images. Since the types of images—red free, color fundus photography screening is based on retinal fundus images, we consider (CFP), autofluorescence, OCT, non-medical images—AI the first three algorithms to be useful when the photogra- could be used as a cascade of filters. First, we could pher (optometrist, general practitioner, or technician) evaluate what kind of image we have, then the eye (in submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 425 DovePress Zapata et al Dovepress Figure 3 Receiver operating characteristic (ROC) curves from the different CNNs. (A) Image type. (B) Quality assessment. (C) Laterality. (D) AMD. (E) GON. our model it is important to differentiated laterality, we reasons for misclassification is difficulty in differentiating receive images from different cameras, automatically between early stages of AMD and normal retinas. We upload in the platform, not all devices have differentiated believe that it is worthwhile for future research to study file from right or left eye) and if they are of sufficient the classification of AMD stages using AI in CFP, as other quality to be assessed; then we could apply CNNs for authors demonstrate using OCT.33 For the purposes of our different pathologies—in our case AMD and GON. current research, we prefer to identify any stage of AMD in The use of AI could decrease the costs of screening order to create future algorithms for classification. We tools in the near future, allowing mass screening for retinal believe that although early AMD will not threaten visual disease in the general population and not just in known acuity in the short term, it should still be studied and diabetics. The incidence of AMD and glaucoma is currently monitored by the ophthalmologist. In a recent review, increasing around the world. These two diseases are Schmidt-Erfurth et al indicate the limited role of CFP for the second and third most frequent causes for visual impair- advanced stages of AMD compared to OCT screening,29 ment in Europe,32 which is why we based our research on but for a general screening platform it may make sense to them. Other publications on automated detection of refer- start with color fundus photography due to cost considera- able AMD with accuracies of 90% can be found in the tions. Also AI has detected patterns that are invisible to the literature.18,26 With our CNNs, we have achieve 86% accu- human eye in CFP such as gender, age, and smoking status. racy, because we have included all types of AMD, including Detection of advanced AMD in retinal fundus images the early stages; at this point, one of the most important should therefore be studied. submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 426 DovePress Dovepress Zapata et al Screening for glaucoma is much more complicated different types of architecture in a future study in order than doing so for other ophthalmological diseases. to try to predict other retinopathies. Authors have had the best outcomes with a combination One of the limitations of this study is the inclusion of of the slit lamp, OCT retinal nerve fiber layer (RNFL), and multiples images, from multiple devices and photogra- visual field tests, but this combination is not the modality phers, some of which only include 40°, 45°, and 50º of of choice when we want to consider devices that are low the central retina. However, this same limitation could also cost, widely available, and easy to use.29 In the latter case, be a strength for the resulting algorithms, which could be we are limited to detecting suspected glaucoma by visua- more reliable for use in the real world without having to lizing only the optic disc, similar to Ting et al, who worry about camera brands or types. The only significant obtained a 94% accuracy for possible glaucoma in barrier would be the assessable AI filter. a dataset of 125,000 diabetic patients.15 In our case, with From an ethical and legal point of view, AI is far from only 4720 images in the suspected glaucoma algorithm, being reliable for general screening for retinal diseases. AI we achieved levels of accuracy of over 80%. Further can detect DR, AMD, or suspected glaucoma perfectly, but research with a better-defined glaucoma database would we have to see what would happen— and who is respon- be desirable in order to reach the maximum potential for sible in the event the algorithm fails—when the system AI in screening for this disease. evaluates an image with uncommon findings, such as We believe the kind of technology discussed in this dystrophies, tumors, or minor infections. In our opinion, paper could be used not only for prospective purposes, but this problem could be solved with bigger datasets that also in retrospective studies, for classification in non-tagged include rare diseases. In the end, the more we teach the big datasets such as hospitals or ophthalmological clinics. AI system, the more it learns. Meanwhile, AI could be We use AI for classification, but one of the most exciting used to ameliorate the cost-effectiveness of using hybrid types of future implementation of CNNs could be the pre- systems supervised by a medical expert, raising doctor’s diction of disease progression or clinical data.29 In the near performance beyond unaided human-only abilities, also future, application of big data will require a well-tagged would be desirable for furthers investigations evaluate AI- database, and AI could be very helpful in this arena—for assisted interpretation in other non-retinal specialist per- massive retrospective classifications of image datasets sonnel such as general ophthalmologists or primary care The accuracy of CNNs appears to be related primarily physicians. to two factors. The first factor is the similarity of classes, Deep learning opens an exciting chapter for the future. and the second is the number of well-classified images. AI algorithms seem to be reliable for determining whether Regarding the first factor, it is necessary to explore the use or not an image is CFP. Those algorithms are useful for of deep learning for fine-grained image classification tech- evaluating the quality of the retinal fundus image and also niques, given that it is easy for CNNs to differentiate for determining DR, AMD, and suspected glaucoma. As a horse from a car but the complexity increases when discussed earlier, the next steps will be to continue “feed- considering the classification of similar images. ing the beast,” providing more and more classified images Concerning the second factor, it is easier to obtain high to improve accuracy and creating new CNNs for other levels of accuracy if we have large and well-classified diseases. One of the most exciting challenges ahead is datasets. Although we used data augmentation techniques designing new networks for making the normal/abnormal in order to improve validation accuracy, having a large algorithm over 90% of accuracy. Apart from technical and dataset could be very important for obtaining a robust and ethical limitations, medical doctors must also establish more general prediction model. We believe false positives how to incorporate the algorithm properly into our clinics could be avoided with larger and better-tagged datasets. In and offices in order to control it and ensure a good use of our case, the main cause of false positives and negatives AI technology in medicine. were borderline cases—images with which even retinal specialists have low interobserver agreement. Disclosure From an engineering point of view, we have shown that MAZ is founder and Medical director in Optretina. DRF is it is possible to propose an effective CNN architecture founder and CTO in Optretina. OF is employee in with high validation accuracy and fewer trainable para- Optretina. The authors report no other conflicts of interest meters. We would like to explore the combination of in this work. submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 427 DovePress Zapata et al Dovepress 17. Abramoff MD, Lou Y, Erginay A, et al. Improved automated detec- References tion of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci. 1. Wong WL, Su X, Li X, et al. Global prevalence of age-related 2016;57:5200–5206. doi:10.1167/iovs.16-19964 macular degeneration and disease burden projection for 2020 and 18. Burlina PM, Joshi N, Pekala M, Pacheco KD, Freund DE, 2040: a systematic review and meta-analysis. Lancet Glob Health. Bressler NM. Automated grading of age-related macular degenera- 2014;2(2):e106–e116. doi:10.1016/S2214-109X(13)70145-1 tion from color fundus images using deep convolutional neural 2. Leggio GM, Bucolo C, Platania CB, Salomone S, Drago F. Current networks. JAMA Ophthalmol. 2017;135(11):1170–1176. doi:10.10 drug treatments targeting dopamine D3 receptor. Pharmacol Ther. 01/jamaophthalmol.2017.3782 2016;165:164–177. doi:10.1016/j.pharmthera.2016.06.007 19. Verbraak FD, Abramoff MD, Bausch GCF, et al. Diagnostic accuracy 3. Surendran TS, Raman R. Teleophthalmology in diabetic retinopathy. of a device for the automated detection of diabetic retinopathy in J Diabetes Sci Technol. 2014;8(2):262–266. doi:10.1177/193229 a primary care setting. Diabetes Care. 2019;42(4):651–656. doi:10.23 6814522806 37/dc18-0148 4. Mansberger SL, Gleitsmann K, Gardiner S, et al. Comparing the 20. Varadarajan AV, Poplin R, Blumer K, et al. Deep learning for pre- effectiveness of telemedicine and traditional surveillance in providing dicting refractive error from retinal fundus images. Invest diabetic retinopathy screening examinations: a randomized controlled Ophthalmol Vis Sci. 2018;59(7):2861–2868. doi:10.1167/iovs.18- trial. Telemed J E Health. 2013;19(12):942–948. doi:10.1089/tmj.20 23887 12.0313 21. Zapata MA, Arcos G, Fonollosa A, et al. Telemedicine for a general 5. Chan CK, Gangwani RA, McGhee SM, Lian J, Wong DS. Cost- screening of retinal disease using nonmydriatic fundus cameras in effectiveness of screening for intermediate age-related macular optometry centers: three-year results. Telemed J E Health. 2017;23 degeneration during diabetic retinopathy screening. Ophthalmology. (1):30–36. doi:10.1089/tmj.2016.0020 2015;122(11):2278–2285. doi:10.1016/j.ophtha.2015.06.050 22. Chollet F. Keras; 2015. Available from: https://github.com/fchollet/ 6. De Bats F, Vannier Nitenberg C, Fantino B, Denis P, Kodjikian L. keras. Accessed January 29, 2020. Age-related macular degeneration screening using a nonmydriatic 23. Ferris FL 3rd, Wilkinson CP, Bird A, et al.; Beckman Initiative for digital color fundus camera and telemedicine. Ophthalmologica. Macular Research Classification Committee. Clinical classification of 2014;231(3):172–176. doi:10.1159/000356695 age-related macular degeneration. Ophthalmology. 2013;120 7. Fierson WM; American Academy of Pediatrics Section on (4):844–851. doi:10.1016/j.ophtha.2012.10.036 Ophthalmology; American Academy of Ophthalmology; American 24. He K, Zhang X, Ren S, Sun J. Deep residual learning for image Association for Pediatric Ophthalmology and Strabismus; American recognition. In Proceedings of the IEEE conference on computer Association of Certified Orthoptists. Screening examination of pre- mature infants for retinopathy of prematurity. Pediatrics. 2013;131 vision and pattern recognition; 2016: 770–778. (1):189–195. 25. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the 8. Pérez MA, Bruce BB, Newman NJ, Biousse V. The use of retinal inception architecture for computer vision. In: IEEE conference on photography in nonophthalmic settings and its potential for computer vision and pattern recognition. arXiv. 2016. neurology. Neurologist. 2012;18(6):350–355. doi:10.1097/NRL.0b0 26. Burlina P, Pacheco KD, Joshi N, Freund DE, Bressler NM. 13e318272f7d7 Comparing humans and deep learning performance for grading 9. Frost S, Brown M, Stirling V, Vignarajan J, Prentice D, AMD: a study in using universal deep features and transfer learning Kanagasingam Y. Utility of ward-based retinal photography in stroke for automated AMD analysis. Comput Biol Med. 2017;82:80–86. patients. J Stroke Cerebrovasc Dis. 2017;26(3):600–607. doi:10.10 doi:10.1016/j.compbiomed.2017.01.018 16/j.jstrokecerebrovasdis.2016.11.112 27. Kim SJ, Cho KJ, Oh S, Liu B. Development of machine learning 10. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with models for diagnosis of glaucoma. PLoS One. 2017;12(5):e0177726. deep convolutional neural networks. Adv Neural Inf Process Syst. doi:10.1371/journal.pone.0177726 2012;1097–1105. 28. Li Z, He Y, Keel S, Meng W, Chang RT, He M. Efficacy of a deep 11. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classifica- learning system for detecting glaucomatous optic neuropathy based tion of skin cancer with deep neural networks. Nature. 2017;542 on color fundus photographs. Ophthalmology. 2018;125(8):11 (7639):115–118. doi:10.1038/nature21056 99–1206. doi:10.1016/j.ophtha.2018.01.023 12. Liu Y, Gadepalli K, Norouzi M, et al. Detecting cancer metastases on 29. Schmidt-Erfurth U, Sadeghipour A, Gerendas BS, Waldstein SM, gigapixel pathology images. arXiv. 2018;1806.07064v1. Bogunović H. Artificial intelligence in retina. Prog Retin Eye Res. 13. Rajpurkar P, Irvin J, Zhu K, et al. CheXNet: radiologist-level pneu- 2018;67:1–29. doi:10.1016/j.preteyeres.2018.07.004 monia detection on chest X-rays with deep learning. arXiv. 2017. 30. Moccia S, Banali R, Martini C, et al. Development and testing of a deep 14. Gargeya R, Leng T. Automated identification of diabetic retinopathy learning-based strategy for scar segmentation on CMR-LGE images. using deep learning. Ophthalmology. 2017;124(7):962–969. doi:10. Magma. 2019;32(2):187–195. doi:10.1007/s10334-018-0718-4 1016/j.ophtha.2017.02.008 31. Molina-Casado JM, Carmona EJ, García-Feijoó J. Fast detection of 15. Ting DSW, Cheung CY, Lim G, et al. Development and validation of the main anatomical structures in digital retinal images based on a deep learning system for diabetic retinopathy and related eye intra- and inter-structure relational knowledge. Comput Methods diseases using retinal images from multiethnic populations with Programs Biomed. 2017;149:55–68. doi:10.1016/j.cmpb.2017.06.022 diabetes. JAMA. 2017;318(22):2211–2223. doi:10.1001/jama.2017. 32. Bourne RR, Stevens GA, White RA, et al. Causes of vision loss 18152 worldwide, 1990–2010: a systematic analysis. Lancet Glob Health. 16. Gulshan V, Peng L, Coram M, et al. Development and validation of 2013;1(6):e339–e349. doi:10.1016/S2214-109X(13)70113-X a deep learning algorithm for detection of diabetic retinopathy in 33. Kermany DS, Goldbaum M, Cai W, et al. Identifying medical diag- retinal fundus photographs. JAMA. 2016;316(22):2402–2410. noses and treatable diseases by image-based deep learning. Cell. doi:10.1001/jama.2016.17216 2018;172(5):1122–1131.e9. doi:10.1016/j.cell.2018.02.010 submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 428 DovePress Dovepress Zapata et al Clinical Ophthalmology Dovepress Publish your work in this journal Clinical Ophthalmology is an international, peer-reviewed journal cover- Central and CAS, and is the official journal of The Society of ing all subspecialties within ophthalmology. Key topics include: Clinical Ophthalmology (SCO). The manuscript management system Optometry; Visual science; Pharmacology and drug therapy in eye dis- is completely online and includes a very quick and fair peer-review eases; Basic Sciences; Primary and Secondary eye care; Patient Safety system, which is all easy to use. Visit http://www.dovepress.com/ and Quality of Care Improvements. This journal is indexed on PubMed testimonials.php to read real quotes from published authors. Submit your manuscript here: https://www.dovepress.com/clinical-ophthalmology-journal submit your manuscript | www.dovepress.com Clinical Ophthalmology 2020:14 429 DovePress