Article published online: 2022-09-14 Original article In vivo computer-aided diagnosis of colorectal polyps using white light endoscopy Authors Ana García-Rodríguez 1, Yael Tudela 2, Henry Córdova1, 3, 4, Sabela Carballal 1, 3, 4, Ingrid Ordás1, 3, 4, Leticia Moreira 1, 3, 4, Eva Vaquero 1, 3, 4, Oswaldo Ortiz 1, Liseth Rivero 1, 3, 4, F. Javier Sánchez 2, Miriam Cuatrecasas3, 4, 5, Maria Pellisé 1, 3, 4, Jorge Bernal2, Glòria Fernández-Esparrach1, 3, 4 Institutions ABSTR AC T 1 Endoscopy Unit. Gastroenterology Department. Background and study aims Artificial intelligence is cur- ICMDiM. Hospital Clínic of Barcelona. University of rently able to accurately predict the histology of colorectal Barcelona, Barcelona, Catalonia, Spain polyps. However, systems developed to date use complex 2 Computer Science Department. Autonomous University optical technologies and have not been tested in vivo. The of Barcelona and Computer Vision Center, Barcelona, objective of this study was to evaluate the efficacy of a Catalonia, Spain new deep learning-based optical diagnosis system, ATE- 3 IDIBAPS, Barcelona, Catalonia, Spain NEA, in a real clinical setting using only high-definition 4 CIBEREHD, Spain white light endoscopy (WLE) and to compare its perform- 5 Pathology Department. Hospital Clínic of Barcelona. ance with endoscopists. University of Barcelona, Barcelona, Catalonia, Spain Methods ATENEA was prospectively tested in real life on consecutive polyps detected in colorectal cancer screening submitted 1.11.2021 colonoscopies at Hospital Clínic. No images were discarded, accepted after revision 10.6.2022 and only WLE was used. The in vivo ATENEA’s prediction (adenoma vs non-adenoma) was compared with the predic- tion of four staff endoscopists without specific training in Bibliography optical diagnosis for the study purposes. Endoscopists Endosc Int Open 2022; 10: E1201–E1207 were blind to the ATENEA output. Histology was the gold DOI 10.1055/a-1881-3178 standard. ISSN 2364-3722 Results Ninety polyps (median size: 5 mm, range: 2–25) © 2022. The Author(s). from 31 patients were included of which 69 (76.7 %) were This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying adenomas. ATENEA correctly predicted the histology in 63 and reproduction so long as the original work is given appropriate credit. Contents of 69 (91.3 %, 95 % CI: 82 %–97 %) adenomas and 12 of 21 may not be used for commercial purposes, or adapted, remixed, transformed or (57.1 %, 95 % CI: 34 %–78 %) non-adenomas while endos- built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/) copists made correct predictions in 52 of 69 (75.4 %, 95 % Georg Thieme Verlag KG, Rüdigerstraße 14, CI: 60 %–85 %) and 20 of 21 (95.2 %, 95 % CI: 76 %–100 %), 70469 Stuttgart, Germany respectively. The global accuracy was 83.3 % (95 % CI: 74%– 90 %) and 80 % (95 % CI: 70 %–88 %) for ATENEA and endos- Corresponding author copists, respectively. Glòria Fernández-Esparrach, MD, PhD, Endoscopy Unit, Conclusion ATENEA can accurately be used for in vivo Institut de Malalties Digestives, Hospital Clínic, Villarroel 170, characterization of colorectal polyps, enabling the endos- 08036 Barcelona, Spain copist to make direct decisions. ATENEA showed a global Fax: +34 93 227 93 87 accuracy similar to that of endoscopists despite an unsatis- mgfernan@clinic.cat factory performance for non-adenomatous lesions. colonoscopy has led to a substantial improvement in survival Introduction [2]. Colorectal cancer (CRC) is the third most common cancer in Optical diagnosis aims to predict the histology of a polyp both sexes and the second leading cause of death in the world based on its endoscopic features. This practice could avoid [1]. Screening for CRC and the removal of neoplastic polyps by pathological analysis in several cases and reduce the derived García-Rodríguez Ana et al. In vivo computer-aided … Endosc Int Open 2022; 10: E1201–E1207 | © 2022. The Author(s). E1201 Original article costs. The American Society for Gastrointestinal Endoscopy’s PIVI working group established diagnostic thresholds for real- time implementation of optical diagnosis for diminutive polyps (≤ 5 mm) [3]. However, PIVI criteria have not yet been met in community-based practices or in non-expert hands [4, 5]. In this regard, the European Society of Gastrointestinal Endoscopy (ESGE) emphasizes the importance of being able to ensure and maintain competence in optical diagnosis as well as considering only the proportion of high-confidence diagnosis as a bench- mark [6]. During the past few decades, considerable technological ad- vances have been made in the application of artificial intelli- gence (AI) to medicine. Computer-aided diagnosis (CADx) is a promising solution to overcome human variation in characteri- zation of polyps by providing decision support. In this specific field, CADx approaches based on deep learning (DL) represent an advantage over previous machine learning by combining both the automatic extraction and classification of image char- acteristics using a multilayered system called convolutional neural networks (CNNs) [7]. The quality and design of published CADx systems has varied over time. Initial studies were carried out retrospectively and ▶ Fig. 1 Example of a manually defined region of interest (ROI) de- lineating the polyp using GTCreator in the training phase. were tested ex vivo using selected stored images [8–12]. More recent studies, most of them DL approaches, have been pro- spectively conducted reporting higher accuracy with faster processing times, which allows diagnosis in real time [13–16]. (ROI) delineating the polyp was manually defined by clinicians However, to date, only CADx systems using complex optical using a program called GTCreator ( ▶ Fig. 1) [17]. The extracted technologies as endocytoscopy have been tested in vivo [13], features, ROI, and actual histologic class of the polyp were used that is, while the colonoscopy is being performed, which still to train the system. ATENEA learned to classify images into ade- hinders the adoption of this technology in daily practice. Fur- noma or non-adenoma categories using an 80 % (high confi- thermore, the newest CADx systems are currently using ad- dence) threshold value (predictions with a confidence value vanced imaging modalities, which clearly limits their imple- < 80 % were not considered valid to represent the actual per- mentation worldwide. formance of the system as they could not guarantee a robust The aim of this study was to evaluate the efficacy of a new performance). CADx system based on DL called ATENEA for in vivo optical di- ATENEA was trained and validated with a total of 1049 high- agnosis in consecutive patients using only white light endos- definition white light images of 483 polyps from 354 patients, copy (WLE) and compare its performance with endoscopists. with a maximum of three images of the same polyp (but with a different view or perspective). Images had variable quality but Methods all had a visible mucosal pattern and only blurred images and polyps covered by mucus were excluded. Images containing pa- Development of ATENEA tient data or without histological analysis were also excluded. As a first step, images of any polyp from routine colonoscopies About two-thirds were adenomas and one-third non-adeno- performed at Hospital Clínic of Barcelona from January 2016 to mas, following a similar proportion to what is found in real life, December 2020 were prospectively collected. The images were with a median size of 4.5 mm (range 2 mm-20 mm). These ima- captured with high-definition colonoscopes (CF-HQ180, CF- ges were randomly split into training (n = 837) and validation HQ-185, CF-HQ-190 and EVIS EXERA III videoprocessor, Olym- sets (n = 212), with the condition that all the images of the pus Europe, Hamburg, Germany) using an external computer same polyp were in the same set. with a frame grabber to ensure image acquisition with the high- est quality. Only white light images without magnification or In vivo experiment chromoendoscopy were used. Data from morphology, location, This observational prospective cohort study was performed in and size of the polyp were collected and periodically transfer- Hospital Clinic of Barcelona between January and March 2021 red by an assistant into the database along with the histological and included individuals from the fecal immunochemical test- category of the lesion obtained after pathological analysis. based (cut-off of ≥ 20 μg of hemoglobin/g of feces) organized The development of ATENEA consisted of two stages: fea- population CRC screening program in which all individuals ture extraction and image classification. Feature extraction aged 50 to 69 years are invited to participate. Colonoscopies was made by using a faster region-based CNN with ResNet50 were performed by four staff endoscopists with more than as backbone. For system training purposes, a region of interest 1000 colonoscopies and adenoma detection rates of 47 %, E1202 García-Rodríguez Ana et al. In vivo computer-aided … Endosc Int Open 2022; 10: E1201–E1207 | © 2022. The Author(s). 48%, 51 %, and 55 %, respectively (average in this program and time period was 49 %). No specific training in optical diagnosis was performed for the study purpose. The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Hospital Clínic of Barcelona (HCB2017/0506, 7/18/2017) and informed con- sent was obtained from all patients involved in the study. It was also registered in ClinicalTrials.gov (NCT03775811). All polyps detected and resected with a final pathological re- port were prospectively included regardless of image quality (not centered polyps, blurred or covered by mucus). Only the image that was considered adequate for the prediction (either by the orientation of the polyp within the image or by the proxi- mity for its correct assessment) was selected by the endos- copist during the exploration and was analyzed. The following variables were collected: estimated size (in mm), location (rec- ▶ Fig. 2 Setting in the endoscopy room showing the position of the tum-sigma, descending, transverse, ascending) and morpholo- assistant sitting in front the computer: the endoscopist is blind to gy according to Paris classification [18], which were properly the image displayed in the computer and ATENEA’s output. noted in the colonoscopy report. In real time, ATENEA classified each polyp as an adenoma or non-adenoma and provided the confidence value for each pre- negative predictive value and accuracy with 95 % confidence in- diction. To avoid losing any prediction, we were more flexible tervals were calculated. Comparisons of these metrics between than in train/validation phases and the minimum threshold ATENEA and endoscopists were performed with Chi squared was reduced to 50 %. Values between 50 % and 80 % were con- test. sidered low-confidence predictions. In this phase, the system Receiver operator characteristics (ROC) curves for use of dif- was fully automatic (without delineation of ROI), providing for ferent prediction confidence values to determine ATENEA in each lesion a bounding box and its corresponding histological vivo performance for diagnosis of adenoma were constructed class. Time needed to acquire and process each image to obtain with Matlab. The area under the curve (AUC) and optimal oper- the automatic prediction was of 40 milliseconds (real time). ating point with its sensitivity and specificity and 95 % confi- For the optical diagnosis process, the endoscopists were dence intervals were calculated. asked to categorize each lesion into two categories (adenoma Two-sided P < 0.05 was considered statistically significant. and non-adenoma) based on its surface pattern and to provide All calculations were performed with R version 4.1.2 and Ma- its diagnostic confidence (high vs low) without any time limita- tlab for Windows version 2020. tion. Endoscopists were blinded to the ATENEA output. The de- cision was intuitive and based on their previous experience but without using chromoendoscopy or any of the existing classifi- Results cations. Serrated sessile lesions (SSLs) were included in the non- During the study period, 90 polyps from 31 consecutive pa- adenoma category. ▶ Fig. 2 shows the set-up in the exploration tients were included. Sixty-nine (76.7 %) were adenomas with a room. median size of 5.0 mm (range: 2–25 mm). Characteristics of the polyps are described in ▶ Table 1. Histopathology ATENEA provided an output for all 90 polyps and was able to All polyps were removed using the usual techniques and sent correctly predict the histology in 63 of 69 adenomas (sensitiv- separately for further evaluation to the Pathology Department, ity: 91.3 %, 95 % CI: 82 %–97 %) and 12 of 21 non-adenomas which was the gold standard. The diagnosis of dysplasia in neo- (specificity: 57.1 %, 95 % CI: 34 %–78 %). Endoscopists were plastic polyps was made based on the Modified Vienna Classifi- able to provide a correct optical diagnosis in 52 of 69 adenomas cation [19]. The pathologist was blinded to the predictions of (sensitivity: 75.4 %, 95 % CI: 64 %–85 %) and 20 of 21 non-ade- both endoscopist and ATENEA. nomas (specificity: 95.2 %, 95 % CI: 76 %–100 %), with an accura- cy of 83.3 % (95 % CI: 74 %–90 %) and 80 % (95 % CI: 70 %–88 %) Statistical analysis for CADx and endoscopists, respectively ( ▶ Table 2). All the polyps were globally analyzed, independently of the With respect to diminutive polyps (≤ 5 mm), ATENEA was endoscopist who performed the exploration. The numbers of able to correctly predict the histology in 31 of 35 adenomas polyps that were true positive (adenomatous polyps predicted (sensitivity: 88.2 %, 95 % CI: 73 %–97 %) and 11 of 18 non-ade- to be adenomatous), true negative (non-adenomatous polyps nomas (specificity: 61.1 %, 95 % CI: 36 %–83 %) and endos- predicted to be non-adenomatous), false positive (non-adeno- copists in 20 of 35 adenomas (sensitivity: 58.8 %, 95 % CI: 41%– matous polyps predicted to be adenomatous) or false negative 75 %) and 17 of 18 non-adenomas (specificity: 94.4 %, 95 % CI: (adenomatous polyps predicted to be non-adenomatous) were 73 %–100 %), respectively, with an accuracy of 78.8 % (95 % CI: calculated. Sensitivity, specificity, positive predictive value, 65 %–89 %) and 71.1 % (95 % CI: 57 %–83 %) for ATENEA and García-Rodríguez Ana et al. In vivo computer-aided … Endosc Int Open 2022; 10: E1201–E1207 | © 2022. The Author(s). E1203 Original article ▶ Table 1 Characteristics of the polyps included in the study. endoscopists, respectively (▶ Table 2). Results of prediction for small polyps (< 10 mm) are also shown in ▶ Table 2. In vivo test (n = 90) ATENEA and the endoscopists disagreed in their prediction in 31 of 90 cases (34.5 %). Endoscopists made their prediction Histological type with high confidence in 79 cases (87 %) and with low confidence ▪ Adenoma 69 (76.7 %) in 11. In all these 11 cases, ATENEA made a good prediction ▪ Non-adenoma 21 (23.3 %) (▶ Fig. 3). The ROC curve for ATENEA in vivo performance (▶ Fig. 4) ▪ Hyperplastic 16 showed an AUC of 0.782. The optimal operating point was ▪ SSL 5 achieved by using 74.2 % as the threshold value with a sensitiv- Size (in mm) ity of 86.9 % (95 % CI: 79.9 %–93.9 %) and a specificity of 66.7 % (95 % CI: 56.9 %–76.4 %). ▪ ≤ 5 mm 52 (57.8 %) ▪ 6 mm to < 10 mm 21 (23.3 %) ▪ ≥ 10 mm 17 (18.8 %) Discussion This study demonstrates that ATENEA, a fully automatic optical Location diagnostic system, can accurately predict polyp histology in an ▪ Rectum-sigma 22 (24.4 %) in vivo setting using only white light, which answers one of the ▪ Others 68 (75.6 %) 10 key research questions identified by international experts related to AI implementation in colonoscopy [20]. ATENEA is Morphology particularly useful for identifying adenomatous lesions, show- ▪ 0-Ip 5 (5.5 %) ing its readiness to be used in a clinical environment. ▪ 0-Is 30 (33.3 %) Until now, the majority of CAD systems developed for char- acterizing colorectal polyps that use AI have required advanced ▪ 0-IIa/IIb 55 (61.1 %) optical diagnostic equipment, such as narrow band imaging (NBI) [9–12, 21] and endocytoscopy [8, 22, 23]. However, WLE. which is the most common endoscopic modality, has not been ▶ Table 2 Performance characteristics of ATENEA and the endoscopists for diagnosis of adenoma. TP FP TN FN PPV SENS NPV Spec Accuracy ATENEA, n = 90 63 9 12 6 87.5 % 91.3 % 66.7 % 57.1 % 83.3 % (95 % CI: 78 %–94%) (95 % CI: 82 %– (95 % CI: 41 %– (95 % CI: 34 %–78%) (95 % CI: 74 %– 97%) 87%) 90%) Endoscopists, 52 1 20 17 98.1 % 75.4 % 54.0 % 95.2 % 80.0 % n = 90 (95 % CI: 90 %– (95 % CI: 64 %– (95 % CI: 37 %– (95 % CI: 76 %– (95 % CI: 70 %– 100%) 85%) 71%) 100%) 88%) P value 0.07 0.02 0.55 0.01 0.7 ATENEA diminutive 30 7 11 4 81.1 % 88.2 % 73.3 % 61.1 % 78.8 % polyps (95 % CI: 65 %–92%) (95 % CI: 73 %– (95 % CI: 45 %– (95 % CI: 36 %–83%) (95 % CI: 65 %– ≤ 5 mm, n = 52 97%) 92%) 89%) Endoscopists di- 20 1 17 14 95.2 % 58.8 % 54.8 % 94.4 % 71.1 % minutive polyps (95 % CI: 76 %– (95 % CI: 41 %– (95 % CI: 36 %– (95 % CI: 73 %– (95 % CI: 57 %– ≤ 5 mm, n = 52 100%) 75%) 73%) 100%) 83%) P value 0.27 0.01 0.38 0.04 0.5 ATENEA small 47 9 12 5 83.9 % 90.4 % 70.6 % 57.2 % 80.8 % polyps (95 % CI: 72 %–92%) (95 % CI: 79 %– (95 % CI: 44 %– (95 % CI: 34 %–78%) (95 % CI: 70 %– < 10 mm, n = 73 97%) 90%) 89%) Endoscopists small 36 1 20 16 97.3 % 69.2 % 55.5 % 95.2 % 76.7 % polyps (95 % CI: 86 %– (95 % CI: 55 %– (95 % CI: 38 %– (95 % CI: 76 %– (95 % CI: 65 %– < 10 mm, n = 73 100%) 81%) 72%) 100%) 86%) P value 0.09 0.02 0.46 0.01 0.69 TP, true positive; FP, false positive; TN, true negative; FN, false negative; PPV, positive predictive value; SENS, sensitivity; NPV, negative predictive value; SPEC, specificity. E1204 García-Rodríguez Ana et al. In vivo computer-aided … Endosc Int Open 2022; 10: E1201–E1207 | © 2022. The Author(s). Endoscopists Adenoma ATENEA Adenoma A B C D E F G H I J K L A B C D E F G H I J K L 1 1 2 2 3 3 4 4 5 5 6 6 Non Adenoma Non Adenoma A B C D E F G H I J K L A B C D E F G H I J K L 7 7 8 8 ▶ Fig. 3 ATENEA and endoscopists’ predictions for all polyps. Each circle represents a polyp and colors correspond to a correct prediction (green) and incorrect prediction (red) with high confidence (full circle) or low confidence (half circle). system takes to process an image, whereas the second is relat- 100 ed to where and when the system is applied (in vivo: in the ex- 90 ploration room versus ex vivo: off-line experiments). These 80 concepts are commonly confused in the literature and, in most 70 of the cases, real-time is used instead of in vivo without men- Sensitivity 60 50 tioning the actual processing time [13, 14]. 40 The results obtained in the present experiment showed that 30 ATENEA had better global accuracy than endoscopists, particu- 20 larly for adenomatous lesions. Conversely, clinicians performed 10 better for the non-adenoma category. It has to be pointed out 0 that they had more information than the system because they 0 10 20 30 40 50 60 70 80 90 100 100-Specificity knew the location and size of the lesion (variables for which ATENEA was blind), so this could lead to a major pretest prob- ability of better diagnosis of non-adenomatous lesions when ▶ Fig. 4 Receiver operating characteristic (ROC) curve for different faced with a small or diminutive polyp in the rectosigmoid co- prediction confidence values for ATENEA. The optimal operating point (defined as the point in the curve with better balance of spe- lon. In contrast, ATENEA performance depended solely on the cificity and sensitivity) was achieved by using 74.2 % as the thresh- number and variability of examples from each of the classes in old value. the training set. The lower performance of ATENEA for non-adenomatous le- sions could limit implementation of a “leave in situ” policy. extensively studied in this context yet. For this reason, we used However, it must be stated that the training of the AI system only white light images to allow for widespread use of ATENEA was greatly affected by the reduced number of examples of regardless of the manufacturer and model of endoscope used. this class in the dataset. There are two reasons for this: first, Our group previously developed a hand-crafted predictive the prevalence of non-adenomatous lesions is generally lower model based on extraction of surface patterns (textons) over and second, they are not systematically removed when located white light images. This system was validated in a small dataset in the rectosigmoid colon, making their collection difficult. containing only 225 high-quality images with a diagnostic ac- Hence, it is necessary to enlarge the dataset both in quantity curacy of over 90 % [24]. Unfortunately, the system was not and percentage of examples of the minority class. fully automatic and could not perform in real time and, conse- CADx systems are not intended to replace endoscopists, but quently, it could not be used in an in vivo experiment. In con- rather, to help them in their tasks. As stated by the ESGE review trast, ATENEA can perform in vivo automatic polyp classifica- on advanced endoscopic imaging, the most likely scenario is tion due to its capability of calculating the result in less than that the intelligent systems will be used as a “second reader” 40 ms (real time). It is important to make a difference between to support the endoscopist’s final diagnosis [25]. In this sense, what real-time and in vivo means: the first refers to the time the an important result from our study was that if endoscopists García-Rodríguez Ana et al. In vivo computer-aided … Endosc Int Open 2022; 10: E1201–E1207 | © 2022. The Author(s). E1205 Original article with low confidence had followed the output of the system, all their predictions would have been correct. This shows the po- Conclusions tential of ATENEA to assist clinicians with less expertise. In conclusion, ATENEA, a CADx system based only on WLE data, Similar to endoscopists, AI systems do also provide a confi- is ready to be used for in vivo and real-time characterization of dence value in their predictions using a percentage instead of colorectal polyps, enabling the endoscopist to make direct de- a binary assessment (high versus low). By varying the confi- cisions. ATENEA achieved a global accuracy similar to endos- dence value, we represented different performances of ATENEA copists, despite having lower performance for non-adenoma- using a ROC curve. Our results show that if the confidence value tous lesions. is low, the sensitivity of the system is lower as it provides more outputs, some of them erroneous. Conversely, if we increase Acknowledgments the allowed confidence value, less but more confident outputs This work was supported by the Instituto de Salud Carlos III will be provided, with the risk of missing some of the lesions (PI17/00894 and PI19/01050), co-funded by the European Un- without a prediction. The so-called Simple Optical Diagnosis ion; the Spanish Ministry for Science and Innovation (MCIN) Accuracy or SODA criteria, which were recently published by (ALETHEIA project, PID2020-120311RB-I00 funded by MCIN/ the ESGE Curricula Working Group [26], are more flexible than AEI/10.13039/501100011033); Fundació Marató de TV3 the PIVI criteria and emphasize the importance of not leaving (201932–30); Secretaria d’Universitats i Recerca de la General- any diminutive lesion with advanced neoplasia in situ. In itat de Catalunya (2014-SGR-1470, 2014-SGR-135, SGR-2017– accordance with SODA criteria, we considered sensitivity to be 1669 and SGR-2017–653), and by CERCA Programme/General- the most important outcome of the intelligent system. The itat de Catalunya. Ana García-Rodríguez received a personal AUC value obtained in our study is similar to the 0.84 reported grant from the Societat Catalana de Digestologia. The authors in a recent meta-analysis using only WLE [27] (which is usually thank Dr. Andrés Cárdenas for his English medical writing lower than the AUC in studies using chromoendoscopy or mag- assistance and Roger Borràs for performing the statistical anal- nification techniques). ysis. Unlike other studies in which low-quality images or the known “difficult for AI cases” were not included, the main strength of this study is that it was performed under real clini- Competing interests cal practice conditions and polyps were included regardless of image quality (not centered polyps, blurred or covered by mu- Glòria Fernández-Esparrach was employed on an advisory board of cus). If we had excluded these polyps from the analysis, the Medtronic and is shareholder of MiWEndo Solutions S.L.Míriam Cua- ATENEA performance would have been better. Nevertheless, trecases is shareholder of MiWEndo Solutions S.L.Maria Pellisé has the lack of publicly available annotated datasets does not allow served on clinical advisory boards for Fujifilm Europe; has served on the clinical advisory board and owns share options in MiWendo S.L.; for a fair comparison between ATENEA and other CADx sys- reports speaker fees from Casen Recordati, Fujifilm, Medtronic and tems. In this sense, comparison of metrics in meta-analysis are Olympus Europe; and has received research funding from Fujifilm difficult to understand as the number, quality, and class of the Europe, Casen Recordati and Ziuz. images are different in each study and affect training and test- ing stages of the validation of computational systems. Funding The study has the following limitations. First, the training dataset was small in terms of number of different polyps. To mi- tigate this, we used pre-trained weights from ImageNet [28], Fundació la Marató de TV3 201932-30 Iniciació a la recerca de la Societat Catalana de Digestologia which is a more general dataset and we applied data augmenta- Generalitat de Catalunya CERCA Programme tion operations (color transformation, rotation, horizontal and Departament d'Innovació, Universitats i Empresa, Generalitat de vertical flip) to enlarge its size. The collaboration with other Catalunya 2014-SGR-135,2014-SGR-1470,SGR-2017-1669,SGR- centers and the public availability of other datasets could also 2017-653 be of great use to both enlarge the dataset and perform multi- Instituto de Salud Carlos III PI17/00894,PI19/01050,PID2020- 120611RB-I00 center validations, increasing the robustness of the results. Spanish Ministry for Science and Innovation (MCIN) PID2020- Second, our study did not consider a separate class for SSL due 120311RB-I00 to the low number of examples in the dataset. The problem of SSLs has not adequately been addressed in other previous stud- ies. It is well known that SSLs are neoplastic lesions and there is Clinical trial not an ideal optical method for their characterization [29]. Due to the clinical relevance of SSLs, some studies propose a classi- fication between neoplastic and non-neoplastic lesions instead ClinicalTrials.gov NCT03775811 of using adenoma vs. non-adenoma. Following this logic, if we TRIAL REGISTRATION: Prospective trial at http://www.clinicaltrials. had included all SSL in the same category as adenomas, four of gov/ five SSLs would have been correctly classified. E1206 García-Rodríguez Ana et al. In vivo computer-aided … Endosc Int Open 2022; 10: E1201–E1207 | © 2022. The Author(s). References [15] Horiuchi H, Tamai N, Kamba S et al. Real-time computer-aided diag- nosis of diminutive rectosigmoid polyps using an auto-fluorescence imaging system and novel color intensity analysis software. Scand J [1] Bray F, Ferlay J, Soerjomataram I et al. Global cancer statistics 2018: Gastroenterol 2019; 54: 800–805 GLOBOCAN estimates of incidence and mortality worldwide for 36 [16] Zachariah R, Samarasena J, Luba D et al. Prediction of polyp pathology cancers in 185 countries. CA Cancer J Clin 2018; 68: 394–424 using convolutional neural networks achieves “resect and discard” [2] Levin B, Lieberman DA, McFarland B et al. Screening and Surveillance thresholds. Am J Gastroenterol 2020; 115: 138–144 for the Early Detection of Colorectal Cancer and Adenomatous [17] Bernal J, Histace A, Masana M et al. GTCreator: a flexible annotation Polyps, 2008: A Joint Guideline From the American Cancer Society, tool for image-based datasets. Int J Comput Assist Radiol Surg 2019; the US Multi-Society Task Force on Colorectal Cancer, and the Ameri- 14: 191–201 can College of Radiology. Gastroenterology 2008; 134: 1570–1595 [18] Lambert R, Lightdale CJ. The Paris endoscopic classification of super- [3] Abu Dayyeh BK, Thosani N, Konda V et al. ASGE technology commit- ficial neoplastic lesions: Esophagus, stomach, and colon – Paris, tee systematic review and meta-analysis assessing the ASGE PIVI France November 30 to December 1, 2002. Gastrointest Endosc thresholds for adopting real-time endoscopic assessment of the his- 2003; 58: S3–S43 tology of diminutive colorectal polyps. Gastrointest Endosc 2015; 81: [19] Schlemper RJ, Riddell RH, Kato Y et al. The vienna classification of 502–516 gastrointestinal epithelial neoplasia. Gut 2000; 47: 251–255 [4] Vu HT, Sayuk GS, Hollander TG et al. Resect and discard approach to [20] Ahmad OF, Mori Y, Misawa M et al. Establishing key research ques- colon polyps: real-world applicability among academic and commu- tions for the implementation of artificial intelligence in colonoscopy: nity gastroenterologists. Dig Dis Sci 2015; 60: 502–508 A modified Delphi method. Endoscopy 2021; 53: 893–891 [5] Rees CJ, Rajasekhar PT, Wilson A et al. Narrow band imaging optical [21] Takemura Y, Yoshida S, Tanaka S et al. Computer-aided system for diagnosis of small colorectal polyps in routine clinical practice: The predicting the histology of colorectal tumors by using narrow-band Detect Inspect Characterise Resect and Discard 2 (DISCARD 2) study. imaging magnifying colonoscopy (with video). Gastrointest Endosc Gut 2016; 66: 887–895 2012; 75: 179–185 [6] Dekker E, Houwen BBSL, Puig I et al. Curriculum for optical diagnosis [22] Mori Y, Kudo SE, Chiu PWY et al. Impact of an automated system for training in Europe: European Society of Gastrointestinal Endoscopy endocytoscopic diagnosis of small colorectal lesions: an international (ESGE) Position Statement. Endoscopy 2020; 52: C10 web-based study. Endoscopy 2016; 48: 1110–1118 [7] Lecun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436– [23] Mori Y, Kudo SE, Mori K. Potential of artificial intelligence-assisted 444 colonoscopy using an endocytoscope (with video). Dig Endosc 2018; [8] Misawa M, Kudo SE, Mori Y et al. Characterization of colorectal lesions 30: 52–53 using a computer-aided diagnostic system for narrow-band imaging [24] Sánchez-Montes C, Sánchez FJ, Bernal J et al. Computer-aided predic- endocytoscopy. Gastroenterology 2016; 150: 1531–1532.e3 tion of polyp histology on white light colonoscopy using surface pat- [9] Byrne MF, Chapados N, Soudan F et al. Real-time differentiation of tern analysis. Endoscopy 2019; 51: 261–265 adenomatous and hyperplastic diminutive colorectal polyps during [25] Bisschops R, East JE, Hassan C et al. Advanced imaging for detection analysis of unaltered videos of standard colonoscopy using a deep and differentiation of colorectal neoplasia: European Society of Gas- learning model. Gut 2019; 68: 94–100 trointestinal Endoscopy (ESGE) Guideline – Update 2019. Endoscopy [10] Gross S, Trautwein C, Behrens A et al. Computer-based classification 2019; 51: 1155–1179 of small colorectal polyps by using narrow-band imaging with optical [26] Houwen BBSL, Hassan C, Coupé VMH et al. Definition of competence magnification. Gastrointest Endosc 2011; 74: 1354–1359 standards for optical diagnosis of diminutive colorectal polyps: Euro- [11] Kominami Y, Yoshida S, Tanaka S et al. Computer-aided diagnosis of pean Society of Gastrointestinal Endoscopy (ESGE) Position State- colorectal polyp histology by using a real-time image recognition ment. Endoscopy 2022; 54: 88–99 system and narrow-band imaging magnifying colonoscopy. Gastroin- [27] Lui TKL, Guo CG, Leung WK. Accuracy of artificial intelligence on his- test Endosc 2016; 83: 643–649 tology prediction and detection of colorectal polyps: a systematic re- [12] Chen PJ, Lin MC, Lai MJ et al. Accurate Classification of diminutive view and meta-analysis. Gastrointest Endosc 2020; 92: 11–22.e6 colorectal polyps using computer-aided analysis. Gastroenterology [28] Deng J, Dong W, Socher R et al. ImageNet: a Large-Scale Hierarchical 2018; 154: 568–575 Image Database in 2009 IEEE Conference on Computer Vision and [13] Mori Y, Kudo SE, Misawa M et al. Real-time use of artificial intelligence Pattern Recognition Workshops (CVPR Workshops). Miami, FL: 2009: in identification of diminutive polyps during colonoscopy a prospec- pp. 248–255 doi:10.1109/CVPR.2009.5206848 Corpus ID: 57246310 tive study. Ann Intern Med 2018; 169: 357–366 [29] Bustamante-Balén M, Satorres C, Ramos-Soler D et al. Evaluation of [14] Rodriguez-Diaz E, Baffy G, Lo WK et al. Real-time artificial intelli- the optical criteria for sessile serrated lesions of the colon: A pro- gence-based histologic classification of colorectal polyps with aug- spective study on a colorectal cancer screening population. Endosc mented visualization. Gastrointest Endosc 2021; 93: 662–670 Int Open 2021; 9: E14–E21 García-Rodríguez Ana et al. In vivo computer-aided … Endosc Int Open 2022; 10: E1201–E1207 | © 2022. The Author(s). E1207