European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 https://doi.org/10.1007/s00259-021-05483-0 ORIGINAL ARTICLE A 3D deep learning model to predict the diagnosis of dementia with Lewy bodies, Alzheimer’s disease, and mild cognitive impairment using brain 18F‑FDG PET Kobra Etminani1 · Amira Soliman1 · Anette Davidsson2 · Jose R. Chang1,3 · Begoña Martínez‑Sanchis4 · Stefan Byttner1 · Valle Camacho5 · Matteo Bauckneht6 · Roxana Stegeran7 · Marcus Ressner8 · Marc Agudelo‑Cifuentes4 · Andrea Chincarini9 · Matthias Brendel10 · Axel Rominger10,11 · Rose Bruffaerts12,13,14 · Rik Vandenberghe12,13 · Milica G. Kramberger15 · Maja Trost15,16 · Nicolas Nicastro17 · Giovanni B. Frisoni18 · Afina W. Lemstra19 · Bart N. M. van Berckel20 · Andrea Pilotto21,22 · Alessandro Padovani21 · Silvia Morbelli6,23 · Dag Aarsland24,25 · Flavio Nobili26,27 · Valentina Garibotto28 · Miguel Ochoa‑Figueroa2,7,29 Received: 6 April 2021 / Accepted: 26 June 2021 / Published online: 30 July 2021 © The Author(s) 2021 Abstract Purpose The purpose of this study is to develop and validate a 3D deep learning model that predicts the final clinical diagno- sis of Alzheimer’s disease (AD), dementia with Lewy bodies (DLB), mild cognitive impairment due to Alzheimer’s disease (MCI-AD), and cognitively normal (CN) using fluorine 18 fluorodeoxyglucose PET (18F-FDG PET) and compare model’s performance to that of multiple expert nuclear medicine physicians’ readers. Materials and methods Retrospective 18F-FDG PET scans for AD, MCI-AD, and CN were collected from Alzheimer’s disease neuroimaging initiative (556 patients from 2005 to 2020), and CN and DLB cases were from European DLB Con- sortium (201 patients from 2005 to 2018). The introduced 3D convolutional neural network was trained using 90% of the data and externally tested using 10% as well as comparison to human readers on the same independent test set. The model’s performance was analyzed with sensitivity, specificity, precision, F1 score, receiver operating characteristic (ROC). The regional metabolic changes driving classification were visualized using uniform manifold approximation and projection (UMAP) and network attention. Results The proposed model achieved area under the ROC curve of 96.2% (95% confidence interval: 90.6–100) on predict- ing the final diagnosis of DLB in the independent test set, 96.4% (92.7–100) in AD, 71.4% (51.6–91.2) in MCI-AD, and 94.7% (90–99.5) in CN, which in ROC space outperformed human readers performance. The network attention depicted the posterior cingulate cortex is important for each neurodegenerative disease, and the UMAP visualization of the extracted features by the proposed model demonstrates the reality of development of the given disorders. Conclusion Using only 18F-FDG PET of the brain, a 3D deep learning model could predict the final diagnosis of the most common neurodegenerative disorders which achieved a competitive performance compared to the human readers as well as their consensus. Keywords Artificial intelligence · Deep learning · FDG PET · Alzheimer’s disease · Mild cognitive impairment · Dementia with Lewy bodies Introduction Neurodegenerative dementias have a huge negative impact This article is part of the Topical Collection on Advanced Image on the healthcare systems globally, especially with increas- Analyses (Radiomics and Artificial Intelligence) ing older population. According to the World Health Organi- * Kobra Etminani zation, there are 50 million persons around the world suffer- kobra.etminani@hh.se ing from dementia and 10 million new cases are anticipated Extended author information available on the last page of the article every year [1]. Alzheimer’s disease, which is considered to 13 Vol.:(0123456789) 564 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Fig. 1  Inclusion and exclu- sion criteria for the datasets used. Since Alzheimer’s disease neuroimaging initiative (ADNI) includes a larger set of Alzheimer’s disease (AD), mild cognitive impairment due to AD (MCI-AD), and cognitively normal (CN), we included those that have no artefacts up to 200 cases per each disorder (except dementia with Lewy bodies (DLB) which the European DLB consortium (EDLB) provided) be the most common neurodegenerative disorder, accounts This study introduces a 3D-CNN model that can predict for approximately 60% of all dementia [2]. Dementia with the final clinical diagnosis of CN, MCI due to AD and pat- Lewy bodies (DLB) is another common neurodegenerative terns of some types of dementia which can represent a chal- disorder, accounting for up to 30% of all cases of demen- lenge in their differentiation for the average reader, like AD tia [3] and is often misdiagnosed and unrecognized [4]. and DLB. We hypothesized that a well-designed 3D-CNN Mild cognitive impairment (MCI) is a prodromal form of model could take the advantage of the 3D 18F-FDG PET dementia, defined by cognitive impairment not interfering scans, detect features or patterns in these kinds of patients, with activities of daily living, leading to AD, DLB, or other and match or even provide better results than the experi- degenerative dementias [5, 6]. The diagnosis of such dis- enced human readers, improving the final diagnostic clas- orders is challenging, even for experienced neurologists, sification of individuals. The model interpretation results making the decision of the use of the appropriate treatment indicate specific brain regions which makes the most dis- difficult in some cases. Therefore, physicians use diagnostic criminations among the included neurodegenerative disor- tests such as neurofunctional imaging in order to provide ders that confirm the findings from the clinical studies. more accurate clinical assessments [7]. 18F-FDG PET scans, which measure cerebral glucose metabolism, have been reported as a useful biomarker for the discrimination of the Material and method above-mentioned neurodegenerative disorders [8]. Deep learning (DL) methods have recently gained more Data acquisition popularity in medical image analysis and in specific in neu- rodegenerative diseases [9–11]. This wide recognition is due The retrospective scans were collected from two differ- to its capability to learn complex representations in imaging ent sources (Fig. 1). The anonymized scans from patients data that are not easily detectable by humans [12], dimin- with probable DLB were collected from the European DLB ishing the need of manual feature extraction (compared to (EDLB) Consortium, 1 which has its core laboratory at traditional machine learning techniques) and detecting the Genoa, Italy having the local institutional ethics committee effective features automatically [13]. approvals including the transfer of fully anonymized imag- Most DL models applied in neurodegenerative diseases ing brain 18 F-FDG PET scans. The scans were performed mainly focus on binary [13, 14] or classify multiple stages according to the European Association of Nuclear Medicine of AD from no dementia to moderate AD on 2D scans [9, (EANM) guidelines [16] from February 2005 to September 15]. However, the utility of such models is limited to the AD 2018. Recruited patients were referred to and assessed at population solely, which makes them unable to discriminate outpatient clinics including memory, movement disorders, from non-AD patterns. In addition, it is difficult to validate geriatric medicine, psychiatric, and neurology clinics as pre- their robustness in the presence of non-AD dementias. The viously described in [17]. Given the retrospective nature of proper diagnosis of dementia patients requires going beyond the present study, diagnosis of probable DLB was originally binary classification and at least recognizing the differences among cognitively normal (CN), MCI and other types of dementia, especially the most common ones such as AD and DLB considering the 3D nature of such scans. 1 https://​www.​ge.​infn.​it/​wordp​ress/?​page_​id=​77&​lang=​en. 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 565 made according to diagnostic criteria for probable DLB as The first 10 layers as well as the last 9 layers of each scan defined by [18]. were excluded as they contain very small objects, result- The EDLB also provided several normal cases that we ing in having a 3D volume of (95 × 79 × 60). Since scans added to the CN database. In order to have comparable sam- are from various sites, feature-wise normalization was per- ple sizes with DLB, up to 200 scans with AD, MCI, and formed using image data preprocessing library in Keras,6 CN were downloaded from the Alzheimer’s disease neuro- i.e., intensities of range [0,1]. Particularly, we treated each imaging initiative (ADNI)2 [19] across ADNI-1, ADNI-2, scan as a sequence of 2D images along the axial plane. We ADNI-3, and ADNI-GO (Grand Opportunities) studies from applied feature-wise normalization for each scan separately December 2005 to March 2020. The ADNI was launched in such that each 3D voxel was normalized by subtracting 2003 as a public–private partnership, led by Principal Inves- feature-specific mean then dividing by the feature-specific tigator Michael W. Weiner, MD. The primary goal of ADNI standard deviation per each scan. has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biologi- Model training cal markers, and clinical and neuropsychological assessment can be combined to measure the progression of MCI and The 3D-CNN model is designed with reference to the early AD. Detailed 18 F-FDG PET imaging protocols can be architecture of VGG16 CNN [21] containing 2 convolu- found at ADNI.3 Data regarding the patient’s final diagnosis tional blocks with 4 convolutional layers and a filter of size were downloaded from the ADNI web portal. For each case, 3 × 3 × 3 across all convolution layers (Fig. 2). The model the latest scans were included. ADNI also provides informa- development and training were conducted using Keras tion regarding the conversions from MCI to AD. Therefore, library on a computer with Linux Ubuntu 18.09 operating the MCI cases are confined to MCI-AD where the latest system, one Nvidia Quadro GV100 GPU card with 32 GB of scans of the MCI cases before conversion to AD during the memory, and 36 CPU core Xenon with 128 GB of memory. follow-up period were included in this study. We performed end-to-end training using mini-batches For both datasets, final clinical diagnosis was used as the of size 6 and Adadelta optimizer with 0.01 learning rate ground truth label. Ninety percent of the final dataset (684 for 50 epochs. Dropout layers with 0.5 rate are used as a cases) was used for model training and internal validation. regularization method, forcing the network to learn more The remaining 10% (73 cases) was used as an independent robust features. To prevent the model from overfitting, an test set for the model and comparison of the reader’s clinical early stopping condition was used by monitoring the valida- interpretations. tion loss in order to end the model training when the model performance stops improving (i.e. less than 0.0001 change Data preprocessing in validation loss for 10 epochs). The model training was performed through 20 rounds of The original DICOM/NIFTI formats were used. The PET k-fold cross validation with k ∈ [2, 10] on the training set scans were spatially normalized to match the International and then accuracy is reported with confidence intervals (CI). Consortium of Brain Mapping template [20] and then skull The model with the highest validation accuracy is chosen for stripped using MATLAB R2016a4 and SPM12.5 The proba- further fine-tuning using the training set with a stochastic bility maps of gray matter, white matter, cerebrospinal fluid, gradient descent optimizer, 0.0001 learning rate, and 0.9 bone, and soft tissue/air were extracted. The skull stripping momentum for 50 epochs. was done by retaining the voxels with high probability of being gray matter, white matter, or cerebrospinal fluid while Model interpretation and visualization discarding those likely being bone and soft tissue/air. The normalized and skull stripped scans were then visually To visualize the attention of the network towards a spe- inspected to assess their normalization quality and ensure cific class, we performed an occlusion experiment [22] for that the spatial normalization converged to an acceptable all four classes in the training dataset, where a volume of solution. All the brains were positioned approximately in 6 × 5 × 5 is removed from the normalized scan with a stride the center of the volume. of 2 for all 3 directions. The results show the cross-entropy response of the network given such occluded data as a func- tion of the position of the occlusion box. The assumption is 2 adni.loni.ucla.edu. that when ignoring a relevant region for the correct classi- 3 http://​adni.​loni.​usc.​edu/​metho​ds/​docum​ents/. fication, the cross-entropy response will be high. The maps 4 https://​www.​mathw​orks.​com/​help/​matlab/​relea​se-​notes-​R2016a.​ html. 5 6 https://​www.​fil.​ion.​ucl.​ac.​uk/​spm/​softw​are/​spm12/. https://​keras.​io/​api/​prepr​ocess​ing/​image/. 13 566 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Fig. 2  The 3D convolutional neural networks architecture of the input as a sequence of 2D images obtained along the axial plane from introduced model. The model utilizes 3D 18F-FDG PET scans after 18F-FDG PET scans being normalized via feature-wise normalization. We consider the are then projected using a mosaic of the slices 5 to 54 (to interact with the viewer, and insert their readings includ- create a 7 × 7 grid) on the axial direction and over layered ing their diagnosis (among the four classes) into the portal. with the average brain. To visualize the metabolism pat- Only scans were visible to the readers (unlike natural clinical terns within each clinical diagnosis related to the highlighted situations), the same as what had been used to train the deep brain regions in the occlusion heatmaps, the average normal- learning model. The inter-rater agreement among the four ized brain scan per each class is calculated over all available readers using Fleiss’s kappa [24] is reported. cases. UMAP is a type of dimensionality reduction algorithm Evaluations [23] to project the high-dimensional dataset into a 2-dimen- sional plane for easy visualization while preserving the For the external validation, receiver operating characteristic relative closeness of data points. We used the unsupervised (ROC) curves of the model on the independent test set (i.e., UMAP to visualize (1) the original normalized data and (2) 10% hold-out data) were plotted and the area under the ROC the extracted features by 3D-CNN model (before the clas- curve (AUC) was calculated with 95% CI. sification layer). For each scan in the independent test set, the majority vot- ing of readers was taken as the consensus clinical diagnosis. Clinical interpretations In case of no consensus, the labels are scattered among the annotated labels, e.g., if an AD case is labeled as AD by Four board-certified nuclear medicine physicians, R1, two readers, MCI-AD by one reader, and CN by one reader, R2, R3, and R4, with 16, 13, 8, and 3 years of experience, we calculate it as 0.5, 0.25, and 0.25 for AD, MCI-AD, and respectively, performed independent interpretations of the CN respectively. The sensitivity and specificity of readers’ independent test set (73 cases). The scans were available in performance and their consensus were plotted in the same axial, sagittal, and coronal views illustrated using Papaya.js ROC space. Sensitivity, specificity, precision, F1 score, and volume viewer7 to the readers via a secure portal using their the confusion matrix with discussion on the misdiagnosed given credentials. Readers could log in whenever they want, cases were reported for both the model and the consensus of human readers. Cohen’s kappa was calculated among con- sensus diagnosis and model predicted diagnosis. 7 https://​github.​com/​rii-​mango/​Papaya. 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 567 Model robustness (from EDLB), and 200 CN (156 cases from ADNI and 44 cases from EDLB). In order to investigate the model sensitivity and robustness The average age of the patients was 77.6 years for men to other similar dementia (i.e., something that the model has (between 56 to 92 years old) and 76.2 years for women not been trained for), 18F-FDG PET scans for eight fronto- (between 56 and 96 years old) in the ADNI dataset. In the temporal lobar degeneration (FTLD) cases were downloaded EDLB set, the average age for men was 72.7 (between 48 to from the frontotemporal lobar degeneration neuroimaging 91 years old) and 72.9 for women (between 50 and 86 years initiative (FTLDNI) database. FTLDNI was funded through old). The overall percentage of women in the ADNI set was the National Institute of Aging, and started in 2010. The 35.2% (196 of 556), and in the EDLB set, was 40.2% (81 primary goals of FTLDNI were to identify neuroimaging of 201). modalities and methods of analysis for tracking frontotem- Initially 200 scans (50 per each class) were sampled using poral lobar degeneration (FTLD) and to assess the value of stratified random sampling as the independent test set; but imaging versus other biomarkers in diagnostic roles.8 eventually 73 cases were read by all four readers. The FTLD scans were pre-processed with the same pro- cedure mentioned before and using the proposed 3D-CNN Clinical interpretations model; we plot the UMAP as well as the occlusion maps besides the output of the model for these eight FTLD cases. Fleiss’s kappa among four readers was 0.19 when discrimi- nating between the diagnoses of AD, MCI-AD, DLB, and MMSE‑based classification CN solely based on metabolic patterns, which is considered as a slight agreement [25]. There were 10 cases in which To perform further classification analysis and enhance the there was no majority voting among readers (two AD, two translational potential of the proposed model, a new model CN, five DLB, and one MCI-AD cases). In 8 of these 10 with different split strategies for training and testing datasets cases, the correct clinical diagnosis was among the readers’ was developed using MMSE scores. MMSE score is used to labels, meaning that two readers could diagnose the correct assess changes to patients suffering from dementia such low disorder while the other two voted for another disorder. The score indicates severe dementia while high score indicates consensus accuracy of the readers was 0.57, and it is higher early or mild conditions of dementia. Thus, scans associated than each individual reader. The accuracies of R1, R2, R3, with high MMSE scores can be challenging for diagnosis. and R4 are 0.56, 0.50, 0.46, and 0.39, respectively which are We performed data stratification according to MMSE to positively associated with the readers’ experience. force the model to get trained on severe cases and tested on More detailed readers’ labeling information is provided mild ones. After sorting the cases in each clinical diagnostic in Table 2. The Fleiss’ kappa is also calculated per each class, 80% (439 cases with low MMSE score) were used for disorder to illustrate the inter-rater agreements in detail. Out training and remaining 20% (112 cases having high MMSE of the 24 labels from the four readers for 6 MCI-AD cases, scores in each category) for testing. We trained the model there were 9 (37.5%) CN, 9 MCI-AD, 4 (16.7%) AD, and 2 using KFCV for 10 rounds. Several performance metrics (8.3%) DLB, showing MCI-AD is commonly misdiagnosed including accuracy, ROC curve, AUC, classification results, as CN, which is very common in the clinical interpretation and UMAP visualization of dataset using the new model are [26]. There was only one CN case where all the readers reported. voted for CN, and in the remaining ones, votes were scat- tered mainly among MCI-AD and CN, that explains very low agreement in CN cases. Results Readers had the highest agreement in AD, with 0.21 as the Fleiss’s kappa. In three of 22 AD cases, all four readers Demographics correctly labeled them as AD and in other three cases, three readers converged to AD. Lastly, there was one DLB case Table 1 summarizes the demographics and mini-mental state where there was no agreement among the readers. examination (MMSE) scores of the two datasets used in this Performance metrics for readers are shown in Table 3. study: EDLB and ADNI, as well as the train and test set dis- In general, readers have higher performance metrics in DLB tributions. The dataset consisted of 757 cases including 200 compared to other clinical diagnoses. Readers are perform- AD (from ADNI), 200 MCI-AD (from ADNI), 157 DLB ing very high in ruling out cases having no DLB (100%); all of 52 non-DLB (i.e., MCI-AD, AD, or CN) labels were truly non-DLB. On the contrary, their performance met- 8 For up-to-date information on participation and protocol, please ric for MCI-AD is relatively low, which is in line with the visit http://​memory.​ucsf.​edu/​resea​rch/​studi​es/​nifd. results from [10]. 13 568 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Table 1  Demographics of datasets. (a) Alzheimer’s disease neuroim- and the independent test set (used for model testing and comparison aging initiative (ADNI) and the European DLB consortium (EDLB) to readers) datasets, (b) train set (used for model training and internal validation), a: Datasets ADNI set No. (percentage) Average age (year) Clinical diagnosis No. cases Female sex Male Female MMSE score AD 200 72 (36.0%) 76.7 ± 8.2 (56–92) 74.0 ± 7.8 (56–89) 22.3 ± 3.3 (8–30) MCI-AD 200 62 (31.0%) 78.6 ± 7.0 (59–91) 76.6 ± 6.9 (57–96) 26.0 ± 3.9 (10–30) CN 156 62 (39.7%) 77.5 ± 5.4 (62–89) 78.3 ± 5.0 (64–87) 29.1 ± 1.4 (18–30) All-ADNI 556 196 (35.2%) 77.6 ± 7.1 (56–92) 76.2 ± 7.0 (56–96) EDLB set DLB 157* 59 (37.5%) 73.3 ± 7.2 (53–91) 74.8 ± 6.4 22.4 ± 4.7 (58–86) (5–30) CN 44** 22 (50.0%) 70.1 ± 10.3 (48–84) 67.5 ± 9.2 (50–83) All-EDLB 201 81 (40.2%) 72.7 ± 7.9 (48–91) 72.9 ± 7.9 (50–86) Total 757 277 (36.6%) 76.4 ± 7.6 (48–92) 75.2 ± 7.4 (50–96) b: Train and test sets Train set: No. (percentage) Average age (year) Source (percentage) Clinical diagnosis No. cases (percentage) Female sex Male Female (range) EDLB (range) AD 178 (89%) 67 (37.6%) 76.9 ± 8.3 (56–91) 74.1 ± 7.7 (59–89) 0 (0.0%) MCI-AD 194 (97.0%) 60 (30.1%) 78.6 ± 6.8 (60–91) 76.6 ± 7.0 (57–96) 0 (0.0%) DLB 136 (86.4%) 52 (38.2%) 73.7 ± 7.0 (55–91) 74.8 ± 6.2 (58–86) 136 (100%) CN 176 (88%) 71 (40.3%) 76.5 ± 6.6 (48–89) 75.4 ± 8.2 (50–87) 34 (19.3%) All-train set 684 (90.4%) 250 (36.5%) 76.7 ± 7.4 (48–91) 75.2 ± 7.4 (50–96) 170 (25.2%) Test set: AD 22 (11%) 5 (22.7%) 75.7 ± 8.4 (57–92) 71.6 ± 10.0 (56–82) 0 (0.0%) MCI-AD 6 (3%) 2 (33.3%) 78.0 ± 14.6 (59–91) 75.5 ± 2.1 (74–77) 0 (0.0%) DLB 21***(13.3%) 7 (33.3%) 70.5 ± 8.3 (53–87) 75.5 ± 8.5 (67—86) 21 (100%) CN 24 (12%) 13 (54.1%) 72.0 ± 10.8 75.9 ± 6.1 (65–85) 10 (41.6%) (55–85) All-test set 73 (9.6%) 27 (37.0%) 73.5 ± 9.6 (53–92) 75.9 ± 7.2 (56–86) 31 (42.4%) * 15 cases with no registered MMSE score, where one of them reported language barrier as the reason ** No MMSE score collected for normal cases in EDLB *** One DLB case contained no age-sex information AD, Alzheimer disease; MCI-AD, mild cognitive impairment due to AD; DLB, dementia with Lewy bodies; CN, cognitively normal Model training Evaluations Figure 3 depicts the 95% confidence intervals for training The performance metrics for the trained model is shown and validation accuracy for each k. The highest validation in Table 3, as well as the readers’ performance metrics. accuracy of 78.9% was achieved with 608 samples and the The ROC curves of the model and the readers are shown validation accuracy was computed using the remaining 76 in Fig. 4. The AUC for prediction of DLB, AD, MCI-AD, samples of the training set. and CN was 96.2%, 96.4%, 71.4%, and 94.7% respectively. Both the model and readers are performing higher in DLB 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 569 Table 2  Confusion matrix: labels of readers and the model Readers’ labels for the independent test set (the Actual label AD* MCI-AD* DLB* CN* Sum (no. cases) Fleiss’s kappa highest label ranked by readers AD 36 (40.9%) 27 (30.6%) 6 (6.8%) 19 (21.6%) 88 (22) 0.21 per each disorder (in each row) MCI-AD 4 (16.7%) 9 (37.5%) 2 (8.3%) 9 (37.5%) 24 (6) 0.07 is shown in bold. The diagonal numbers are the true positives DLB 19 (22.6%) 11 (13.1%) 41 (48.8%) 13 (15.5%) 84 (21) 0.04 shown in underlined CN 4 (4.2%) 27 (28.1%) 10 (10.4%) 55 (57.3%) 96 (24) -0.02 Sum 63 (21.6%) 74 (25.3%) 59 (20.2%) 96 (32.9%) 292 (73) 0.19 Model Labels Actual label AD* MCI-AD* DLB* CN* No. cases Cohen’s kappa model vs. con- sensus AD 20 (91%) 1 (4.5%) 0 1 (4.5%) 22 0.13 MCI-AD 3 (50%) 1 (16.7%) 0 2 (33.3%) 6 0.21 DLB 0 (%) 1 (4.8%) 18 (85.7%) 2 (9.5%) 21 0.27 CN 1 (4.2%) 2 (8.3%) 0 21 (87.5%) 24 0.68 Sum 24 (32.9%) 5 (6.8%) 18 (24.7%) 26 (35.6%) 73 0.54 * Numbers in parentheses show the percentage of the whole labels collected for all scans per each disorder AD, Alzheimer disease; MCI-AD, mild cognitive impairment due to AD; DLB, dementia with Lewy bod- ies; CN, cognitively normal Table 3  Performance metrics Metric Sensitivity * Specificity * Precision * F1 score No. cases for the proposed deep learning model and the consensus Consensus of the readers of the readers (bold values AD 0.47 (10.5/22) 0.90 (46.25/51) 0.68 (10.5/15.25) 0.56 22 are the highest between readers consensus vs. model MCI-AD 0.25 (1.5/6) 0.75 (50.25/67) 0.08 (1.5/18.25) 0.12 6 performance, underlined values DLB 0.63 (13.25/21) 1.0 (52/52) 1.0 (13.25/13.25) 0.77 21 illustrate similar performance) CN 0.70 (17/24) 0.81 (39.75/49) 0.64 (17/26.25) 0.67 24 Model AD 0.91 (20/22) 0.92 (47/51) 0.83 (20/24) 0.87 22 MCI-AD 0.17 (1/6) 0.94 (63/67) 0.20 (1/5) 0.18 6 DLB 0.86 (18/21) 1.0 (52/52) 1.0 (18/18) 0.92 21 CN 0.88 (21/24) 0.90 (44/49) 0.81 (21/26) 0.84 24 * Numbers in parentheses are the number of cases (raw data) used to calculate the metric In case of no consensus, the labels are scattered among the annotated labels, e.g. if an AD case is labeled as AD by two readers, MCI-AD by one reader, and CN by one reader, we calculate it as 0.5, 0.25, and 0.25 for AD, MCI-AD, and CN respectively AD, Alzheimer disease; MCI-AD, mild cognitive impairment due to AD; DLB, dementia with Lewy bod- ies; CN, cognitively normal cases and lower in MCI cases. The model reached a per- R1 has higher sensitivity in diagnosing CN, while R3 has fect performance in DLB cases with 86% sensitivity (18 higher specificity in the same disorder. out of 21 DLB cases were detected), 100% specificity (52 Cohen’s kappa among consensus diagnosis and model non-DLB cases by the model were correctly ruled out), predicted label was 0.54, which is considered as moderate and 100% precision (18 cases labeled as DLB were cor- agreement. Among 13 misdiagnosed cases by the model, rectly classified), and F1 score 92%. The proposed model there were 5 MCI-AD cases, 2 AD, 3 CN, and 3 DLB cases performed better than all the readers and their consensus, as illustrated in Table 2. We looked into these cases and in some cases even statistically significant. As depicted in compared them to the readers’ labeling. In MCI-AD cases, Fig. 4, some readers have higher sensitivity in some dis- one case was correctly diagnosed by both the model and the orders while others have higher specificity. For instance, readers, and the remaining 5 cases, all were misdiagnosed 13 570 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Fig. 3  Confidence intervals (95% CI) for training and vali- dation accuracy during k-fold cross validation. KFCV, k-fold cross validation Fig. 4  Receiver operating char- acteristic (ROC) curves with 95% confidence interval (CI) on the test set for a Alzheimer disease (AD), b mild cognitive impairment due to AD (MCI- AD), c dementia with Lewy bodies (DLB), and d cognitively normal (CN). (R-All indicates the consensus labeling among the four readers, R1-4, reader 1–4) (a) Model vs Readers (AD) (b) Model vs Readers (MCI-AD) (c) Model vs Readers (DLB) (d) Model vs Readers (CN) both by the consensus of the readers and the model, that these 13 model-misdiagnosed cases, 11 cases were also con- explains the high agreement between consensus of the read- sensus-misdiagnosed, and 6 cases were similarly consensus- ers and the model (Cohen’s kappa 0.68). In total, among misdiagnosed to the same disorder. 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 571 Fig. 5  Uniform manifold approximation and projection (UMAP) visualization of the data: a shows the visualization of the original input data and b shows the map of the features extracted from the proposed model, the layer before the classifier layer. The red circles denote the misclassified testing samples. (AD, Alzheimer dis- ease; MCI-AD, mild cognitive impairment due to AD; DLB, dementia with Lewy bod- ies; CN, cognitively normal; UMAP-D, uniform manifold approximation and projection dimension) (a) 2D visualization of data (train/test) (b) 2D visualization of data representation (train/test) after model training Model interpretation and visualization or in the borders of two classes. It is worth mentioning again that the ground truth labels for the whole dataset As shown in UMAP visualizations in Fig. 5b, DLB cases (both the ADNI and EDLB) are the final clinical diagnosis were separated compared to Fig. 5a and it is explaining the obtained from these sources and we are not aware if nec- good performance of the model due to the relevant extracted ropsy evaluation has been performed in any of those cases. features. The other interesting pattern in this figure is the The results of the occlusion experiment which indi- distribution of cases from CN to MCI-AD and then to AD, cated the network attention are illustrated in Fig. 6. The which is happening also in reality. DLB cases are very well highlighted regions in each disorder indicate which brain separated and cases from CN to AD are spread from CN to regions were of more attention in the proposed model in its MCI-AD and to AD, which explains the development of prediction. In AD (Fig. 6a), the posterior cingulate cortex AD. The extracted features of the proposed model were able is the most discriminating region, slightly along with the to separate these four classes well enough, although using temporal lobes and the anterior cingulate cortex. In MCI- unsupervised UMAP. AD (Fig. 6b), the most discriminating regions are similar The test cases with red circles in Fig. 5b are the mis- to AD with more emphasis on the posterior cingulate cor- classified cases by the model. They are mainly those cases tex, the middle temporal gyrus, gyrus rectus/orbital gyri, that have happened to be in the middle of the wrong class and also on the parieto-occipital cortex. Furthermore, the 13 572 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 (a) AD (b) MCI-AD (c) DLB (d) CN 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 573 ◂Fig. 6  Results of the occlusion experiments for a Alzheimer disease MMSE‑based classification (AD), b mild cognitive impairment due to AD (MCI-AD), c dementia with Lewy bodies (DLB), and d cognitively normal (CN). The results are projected by creating a mosaic of slices in the axial direction. The The new model trained on low MMSE scores achieved best cross-entropy maps have been over layered with the average brain accuracy 80%, 82%, 66% for training, validation, and test- ing accuracy, respectively. The validation accuracy is higher than the validation accuracy of the proposed model trained posterior cingulate cortex is also taking an important role without MMSE stratification, while the testing accuracy is in differentiating DLB cases (Fig. 6c) besides the occipital lower. The decay in performance is expected, due to the cortex. And finally, in CN (Fig. 6d), the occipital cortex, stratification that forces the model to get trained on easy the cerebellum, and slightly postcentral gyrus and striatum cases and tested on hard cases to predict. The test set con- are the highlighted regions. tained 112 cases, out of which 33 were misclassified. Fig- The posterior cingulate cortex is important for all ure 9a depicts the UMAP visualization of the new trained the given neurodegenerative disorders, i.e., AD, MCI- model, which conveys the same pattern as the UMAP visu- AD, and DLB, and not in CN. It shows the pattern in alization of the random split model shown in Fig. 5b with this brain region makes the most difference in a cog- less clear borders that justifies the lower performance. The nitively normal brain compared to dementia-involved ROC curves and AUCs are shown in Fig. 9b. Compared to ones. the random split results, no change in DLB and MCI-AD is The average over all normalized brain scans for each observed, but AD and CN experienced a drop of 5 and 10% clinical diagnosis is illustrated in Fig. 7. AD (Fig. 7a) and in AUC respectively. MCI-AD (Fig. 7b) share similar metabolism patterns with Figure 10 illustrates the MMSE scores of the classifi- MCI-AD in the highlighted regions as shown in Fig. 6b. The cation results both in the random split (Fig. 10a) which is hypometabolism pattern in the posterior cingulate cortex dif- performed in design of the proposed model, and the stratified fers the most among the different disorders as expected from split where the same model is trained on low MMSE scores, Fig. 6. and tested on high MMSE scores (Fig. 10b). In the random split, the misclassification errors happened both in low and Model robustness high MMSE scores in CN and MCI-AD groups, while in AD and DLB, the few misclassified test cases occurred close to Among the eight FTLD cases, three cases were classi- the high MMSE scores. However, in the stratified split, the fied as CN while the remaining five cases were classi- misclassified test cases are not different from the correctly fied as AD by the model. Figure 8a shows the UMAP classified test cases in terms of MMSE score. representation of the training set (similar to Fig. 5b with test cases excluded) where FTLD cases are also plotted. All the eight cases are mapped close to each other in the Discussion UMAP space. Interestingly, the generated representation reflects the similarity of FTLD cases with CN and AD Today, nuclear medicine specialist physicians make pat- cases and not to DLB. What is expected to observe in tern recognition decisions on FDG PET scans using visual FTLD is low FDG uptake in the frontal and temporal and qualitative readings, which is complex and challenging lobes [29]. A patient with a chronic AD can eventually and needs years of experience. In this study, we proposed have involvement of the frontal lobes and look like a a 3D-CNN model to predict the diagnosis based on 18F- FTLD. These five FTLD cases are very probable to have FDG PET scans. The datasets were achieved from ADNI involvement not only of the frontal and temporal lobes and EDLB. The performance of the model was shown to be but even the parietal lobes might be involved. We per- robust across the studied disorders: DLB, AD, MCI-AD, and formed the occlusion experiments using FTLD cases to CN and also in comparison to the nuclear medicine read- investigate further the highlighted regions in these brain ers. The proposed model reached a competitive performance scans. As shown (see Fig. 8.b and 8.c), though there compared to an experienced reader and also the consensus is a huge overlap in highlighted regions with previous of them. With further validation with more diverse datasets results of AD/CN, FTLD cases show different intensities. and extending to include more similar disorders, the pro- Hence, the learned patterns by the model correspond to posed model can be used as an augmentation to the provided the metabolism patterns within each disorder. 18F-FDG PET software, improving the diagnosis of such 13 574 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 (a) AD (b) MCI-AD (c) DLB (d) CN Fig. 7  The average brain scan for all normalized cases with a Alzheimer disease (AD), b mild cognitive impairment due to AD (MCI-AD), c dementia with Lewy bodies (DLB), and d cognitively normal (CN) neurodegenerative disorders, especially in situations where MCI as a generic cognitive impairment. Second, the there is missing presence of experienced physicians. test set was relatively big (n = 73) compared to the This study has several strengths and contributions similar studies, for example (n = 40) within the inde- that are briefly explained here. First, the dataset is rel- pendent test set as reported in [10] and (n < 35) with atively diverse since it contains cases from ADNI and tenfold cross validation as reported in [27]. The CN EDLB sources, in specific the CN cases were a combi- cases in the test set were selected balanced from the nation of both sources and DLB cases were from 7 dif- two sources ADNI and EDLB not to be biased towards ferent sites in Europe. The MCI cases were confined ADNI which is more dominant in providing our CN to those that further developed AD and not including cases. 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 575 (a) UMAP visualization of data representation (train) after model training and FTLD cases marked with red circles. (b) Occlusion results of FTLD cases predicted as CN (c) Occlusion results of FTLD cases predicted as AD Fig. 8  Results of FTLD cases, a the representation space with training dataset and adding new 8 cases with FTLD, b occlusion results of FTLD cases labeled as CN, and c occlusion results for FTLD cases labeled as AD Third, the dataset includes DLB, a non-AD disorder, as And finally, the extracted patterns from the interpreta- well as the AD family (AD and MCI-AD), and CN to make tion of the model show that the posterior cingulate cortex it more robust in the predicted diagnosis compared to similar is playing an important role in discriminating these neuro- studies which tried to discriminate different stages of AD, degenerative disorders, i.e., AD, MCI-AD, and DLB, and make the utility of their algorithm limited to AD patient not in CN. It shows the pattern in this brain region makes population only. the most difference in a cognitively normal brain compared 13 576 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Fig. 9  Results of training a new model with MMSE-based data split, a the UMAP visualization of training/testing datasets with misclassified test cases identi- fied with red circles, b receiver operating characteristic (ROC) curve for model predictions with testing dataset (a) 2D visualization of data representation (train/test) after model training (b) ROC-Curve for testing data set to dementia-involved ones. It also depicted AD and MCI- with the diagnosis of multiple neurodegenerative dis- AD sharing the same affected brain regions. While in [10], orders. Differently from existing methods, the pre- model visualization with saliency map did not reveal a sented model can distinguish cases of AD, CN, MCI- human interpretable imaging biomarker that appears influ- AD, and DLB with AUC of 96.4%, 94.7%, 71.4%, and ential for AD prediction. 96.2%, respectively. The model robustness test over Recently, substantial work in the area of applying DL a few FTLD cases (which was not part of the train- methods has been done for the classification of differ- ing process), revealed that the learned metabolism by ent brain disorders [9–11]. However, most of the work the model are relevant and consistent to the expected has been performed using structural imaging of the brain patterns. and very little work has been presented by applying DL, One of the limitations of the study was that all AD particularly CNNs, using functional imaging, specifically and MCI-AD disorders were obtained from ADNI, which 18F-FDG PET scans. In Table 4, we summarized the makes the robustness of the proposed model in these two results obtained by recent DL studies for the diagnosis of cases limited to the clinical distribution of ADNI data- AD and MCI using FDG PET [10, 14, 27, 28]. sets. Furthermore, the number of MCI-AD cases in the In summary, the proposed model takes the advantage independent test set was small. Performance dropped of the 3D 18F-FDG PET scans and provides high pre- somewhat for the classification of MCI-AD cases, but dictive performance as well as strong generalizability this was analogous to the performance of the readers. 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 577 Fig. 10  MMSE-based clas- sification results: a random split, b stratified split where low MMSE scores are used for training and high ones for test (a) random split (b) stratified split Also, FDG PET may be normal in the MCI-AD stage workflow as a decision support tool. We will look into how where the diagnosis heavily relies on fluid biomarkers. to add more explanations to the outcome of the provided The second limitation is that the proposed model pre- model to increase transparency and trust. dicted the diagnosis based on 18F-FDG PET scans only, the same with the human readers in this study. But, in real practice, clinicians make the final decision based on sev- Funding Open access funding provided by Halmstad University. This study was part of a collaborative project between Center for Applied eral other clinical evaluations. We believe if other clinical Intelligent System Research (CAISR) at Halmstad University, Sweden, evaluations of the patients are added to the model, the and Department of Clinical Physiology, Department of Radiology and performance will even reach higher values. On the other the Center for Medical Imaging Visualization (CMIV) at Linköping hand, the proposed model is able to be embedded to the University Hospital, Sweden, and the European DLB consortium, which was funded by Analytic Imaging Diagnostics Arena (AIDA) 18F-FDG PET software devices that nuclear medicine spe- initiative, jointly supported by VINNOVA (Grant 2017–02447), cialists are normally using without any extra patient infor- Formas and the Swedish Energy Agency. VG was supported by mation needed and still be able to discriminate among sev- the Swiss National Science Foundation (projects 320030_169876, eral neurodegenerative disorders with high performance. 320030_185028) and the Velux Foundation (project 1123). RB is a senior postdoctoral fellow of the Flanders Research Foundation (FWO Third, we only include DLB as a non-AD disorder. It is 12I2121N). worth trying to include more neurodegenerative disorders to check the robustness of the algorithm in the presence of Data availability Part of data collection and sharing for this project was other similar diseases. funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) One of the future works alongside with providing solu- (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12–2-0012). tions to the above-mentioned limitations will be to inves- tigate integrating the proposed algorithm into clinical 13 Table 4  Summary of the state-of-the-art studies applying deep learning (DL) using 18F-FDG PET for diagnosis of Alzheimer’s disease (AD) and mild cognitive impairment (MCI). We report 578 the performance of DL models using classification accuracy (ACC), sensitivity (SEN), specificity (SPE), and area under the curve (AUC) Reference Summary Disorders Dataset specifi- ACC​ SEN SPE AUC​ 13 cations Ding et al. This work AD, MCI, non- ADNI (ADNI) AD 2019 [10] presented AD/MCI 2019 scans - 81% 94% 92% the success including: (ADNI) MCI of adopt- 484 AD ing transfer 861 MCI - 54% 68% 63% learning 764 non-AD/ (ADNI) Non-AD/MCI for medical MCI - 59% 75% 73% imaging by Authors’ Insti- (AI) AD using Incep- tution tionV3 model 40 imag- - 100% 82% 98% pre-trained ing studies (AI) MCI on ImageNet. including: – 43% 58% 52% The model 7 AD (AI) Non-AD/MCI was fine- 7 MCI tuned using 26 non-AD/ – 35% 93% 84% 18F-FDG MCI PET scans from ADNI (1921 imag- ing studies). However, the introduced method transformed 3D 18F-FDG PET scans into 2D grids which pre- vented gener- ating human interpretable imaging to reveal model decisions for AD predic- tion European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Table 4  (continued) Reference Summary Disorders Dataset specifi- ACC​ SEN SPE AUC​ cations Huang et al. A 3D VGG AD, MCI, AD ADNI (18 F-FDG PET) CN vs. AD 2019 [14] network was 2145 18F-FDG PET scans 89.1% 90.2% 87.8% 92.7% introduced to predict including: (T1-MR) CN vs. AD the develop- 647 AD ment of AD 731 CN 81.2% 79.3% 83.5% 83.7% by process- 767 MCI (Multi-modality) CN vs. AD ing multi- modality 90.1% 90.9% 89.2% 90.8% information (Multi-modality) CN vs. pMCI via usage of both T1-MR 87.5% 90.8% 80.6% 87.6% and 18F-FDG PET images. The results showed a slight improve- ment in performance of classify- European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 ing AD, MCI, and CN patients. For sake of comparison with other methods, we reported the results of the models that were trained using a single imaging source as well as multi- modality information 13 579 Table 4  (continued) 580 Reference Summary Disorders Dataset specifi- ACC​ SEN SPE AUC​ cations 13 Liu et al. 2018 The model AD, MCI, CN ADNI AD vs. CN [27] provided the 339 subjects including: 91.2% 91.4% 91% 95.3% classifica- tion using 93 AD MCI vs. CN 2D CNN and 146 MCI recurrent neu- 100 CN 78.9% 78.1% 80% 83.9% ral networks (RNNs). The model treated 3D 18 F-FDG PET as a sequence of 2D images. Adopted DL methods captured the intra-slice and inter- slice features for the classification task. How- ever, con- structing 2D sequences represent- ing differ- ent views increases the complexity of used DL models, thus limits the applicabil- ity of such framework on large datasets European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Table 4  (continued) Reference Summary Disorders Dataset specifi- ACC​ SEN SPE AUC​ cations Shen et al. A deep belief MCI-AD, ADNI MCI-AD vs. sMCI 2019 [28] network sMCI 109 18 F-FDG PET scans for 86.6% 89.5% 82.5% 90.9% was used to extract MCI cases feature including: representa- 47 MCI-AD tion of some 62 sMCI extracted regions of the brain by employing region- growing algorithm. Additionally, a support vec- tor machine classifier was used to iden- tify patients in MCI European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 who would progress on to AD (MCI- AD) and discriminate these from patients with stable MCI conditions (i.e., sMCI) DL, deep learning; CN, cognitively normal; MCI-AD, mild cognitive impairment due to AD; sMCI, stable MCI; ADNI, Alzheimer’s disease neuroimaging initiative 13 581 582 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 Declarations 9. Ramzan F, Khan MUG, Rehmat A, Iqbal S, Saba T, Rehman A, et al. A deep learning approach for automated diagnosis and multi- class classification of Alzheimer’s disease stages using resting- Ethics approval This research study was conducted retrospectively state fMRI and residual neural networks. J Med Syst. 2019;44:37. from (EDLB) data obtained for clinical purposes. Local institutional 10. Ding Y, Sohn J, Mg K, H T, R H, Nw J, et al. A deep learning ethics committee approvals for the retrospective analyses were availa- model to predict a diagnosis of Alzheimer disease by using 18F- ble for all centers in Europe, including the transfer of fully anonymized FDG PET of the brain. Radiology. 2018;290:456–64. imaging data. 11 Choi H, Kim YK, Yoon EJ, Lee J-Y, Lee DS, Alzheimer’s dis- ease neuroimaging initiative. Cognitive signature of brain FDG Consent to participate Regarding the data from Linköping’s University PET based on deep learning: domain transfer from Alzheimer’s Hospital, informed consent was waived for this retrospective assess- disease to Parkinson’s disease. Eur J Nucl Med Mol Imaging. ment and additionally, all patients were informed by letter that their 2020;47:403–12. medical data can be rendered anonymous and used for scientific pur- 12. Katako A, Shelton P, Goertzen AL, Levin D, Bybel B, Aljuaid M, poses. All patients from the rest of the centers gave informed written et al. Machine learning identified an Alzheimer’s disease-related consent for the imaging procedure and radiopharmaceutical applica- FDG-PET pattern which is also expressed in Lewy body dementia tion. and Parkinson’s disease dementia. Sci Rep. 2018;8:13236. 13. Al-Shoukry S, Rassem TH, Makbol NM. Alzheimer’s diseases Conflict of interest The authors declare that they have no conflicts of detection by using deep learning algorithms: a mini-review. IEEE interest. Access. 2020;8:77131–41. 14. Huang Y, Xu J, Zhou Y, Tong T, Zhuang X, Initiative (ADNI) Open Access This article is licensed under a Creative Commons Attri- the ADN. Diagnosis of Alzheimer’s Disease via Multi-Modality bution 4.0 International License, which permits use, sharing, adapta- 3D Convolutional Neural Network. Front Neurosci [Internet]. tion, distribution and reproduction in any medium or format, as long Frontiers; 2019 [cited 2021 Apr 6];13. Available from: https://​ as you give appropriate credit to the original author(s) and the source, www.​f ront​iersin.​org/​artic​les/​10.​3389/​fnins.​2019.​00509/​full. provide a link to the Creative Commons licence, and indicate if changes Accessed Jan-March 2021. were made. The images or other third party material in this article are 15. Mehmood A, Maqsood M, Bashir M, Shuyuan Y. A Deep Sia- included in the article’s Creative Commons licence, unless indicated mese Convolution Neural Network for Multi-Class Classification otherwise in a credit line to the material. If material is not included in of Alzheimer Disease. Brain Sci Multidisciplinary Digital Pub- the article’s Creative Commons licence and your intended use is not lishing Institute. 2020;10:84. permitted by statutory regulation or exceeds the permitted use, you will 16. Varrone A, Asenbaum S, Vander Borght T, Booij J, Nobili F, need to obtain permission directly from the copyright holder. To view a Någren K, et al. EANM procedure guidelines for PET brain imag- copy of this licence, visit http://​creat​iveco​mmons.​org/​licen​ses/​by/4.​0/. ing using [18F]FDG, version 2. Eur J Nucl Med Mol Imaging. 2009;36:2103–10. 17. Kramberger MG, Auestad B, Garcia-Ptacek S, Abdelnour C, Olmo JG, Walker Z, et al. Long-term cognitive decline in demen- References tia with Lewy bodies in a large multicenter, international cohort. J Alzheimers Dis. 2017;57:787–95. 1. Dementia Key Facts [Internet]. WHO. 2020 [cited 2021 Apr 6]. 18. McKeith IG, Dickson DW, Lowe J, Emre M, O’Brien JT, Feld- Available from: https://​www.​who.​int/​news-​room/​fact-​sheets/​ man H, et al. Diagnosis and management of dementia with detail/​demen​tia. Accessed Jan-March 2021. Lewy bodies: third report of the DLB Consortium. Neurology. 2. Ferri CP, Prince M, Brayne C, Brodaty H, Fratiglioni L, Ganguli 2005;65:1863–72. M, et al. Global prevalence of dementia: a Delphi consensus study. 19. Mueller SG, Weiner MW, Thal LJ, Petersen RC, Jack C, Jagust Lancet. 2005;366:2112–7. W, et al. The Alzheimer’s disease neuroimaging initiative. Neu- 3. Zaccai J, McCracken C, Brayne C. A systematic review of preva- roimaging Clinics of North America. 2005;15:869–77. lence and incidence studies of dementia with Lewy bodies. Age 20 Mazziotta JC, Toga AW, Evans A, Fox P, Lancaster J. A probabil- Ageing. 2005;34:561–6. istic atlas of the human brain: theory and rationale for its develop- 4. McKeith I. Dementia with Lewy bodies. Dialogues Clin Neurosci. ment .The International Consortium for Brain Mapping (ICBM). 2004;6:333–41. Neuroimage. 1995;2:89–101. 5. Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox 21. Simonyan K, Zisserman A. Very deep convolutional networks NC, et al. The diagnosis of mild cognitive impairment due to Alz- for large-scale image recognition. arXiv:14091556 [cs] [Inter- heimer’s disease: recommendations from the National Institute on net]. 2015 [cited 2021 Apr 6]; Available from: http://​arxiv.​org/​ Aging-Alzheimer’s Association workgroups on diagnostic guide- abs/​1409.​1556. Accessed Jan-March 2021. lines for Alzheimer’s disease. Alzheimers Dement. 2011;7:270–9. 22. Zeiler MD, Fergus R. Visualizing and understanding convolu- 6. Dubois B, Feldman HH, Jacova C, Cummings JL, Dekosky ST, tional networks. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, Barberger-Gateau P, et al. Revising the definition of Alzheimer’s editors. Computer Vision – ECCV 2014. Cham: Springer Inter- disease: a new lexicon. Lancet Neurol. 2010;9:1118–27. national Publishing; 2014. p. 818–33. 7. Chételat G, Arbizu J, Barthel H, Garibotto V, Law I, Morbelli S, 23. McInnes L, Healy J, Melville J. UMAP: uniform manifold approx- et al. Amyloid-PET and 18F-FDG-PET in the diagnostic investi- imation and projection for dimension reduction. arXiv:180203426 gation of Alzheimer’s disease and other dementias. Lancet Neurol [cs, stat] [Internet]. 2020 [cited 2021 Apr 6]; Available Elsevier. 2020;19:951–62. from: http://a​ rxiv.o​ rg/a​ bs/1​ 802.0​ 3426. Accessed Jan-March 2021. 8. Nobili F, Arbizu J, Bouwman F, Drzezga A, Agosta F, Nestor 24. Fleiss JL. Measuring nominal scale agreement among many P, et al. European Association of Nuclear Medicine and Euro- raters. Psychol Bull US: American Psychological Association. pean Academy of Neurology recommendations for the use of 1971;76:378–82. brain 18F-fluorodeoxyglucose positron emission tomography in 25. Nichols TR, Wisner PM, Cripe G, Gulabchand L. Putting the neurodegenerative cognitive impairment and dementia: Delphi kappa statistic to use. Qual Assur J. 2010;13:57–61. consensus. Eur J Neurol. 2018;25:1201–17. 13 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 583 26. Drzezga A, Grimmer T, Riemenschneider M, Lautenschlager N, belief network based on 18F-FDG-PET images. Mol Imaging. Siebner H, Alexopoulus P, et al. Prediction of individual clinical SAGE Publications Inc; 2019;18:1536012119877285. outcome in MCI by means of genetic assessment and (18)F-FDG 29. Brown RK, Bohnen NI, Wong KK, Minoshima S, Frey KA. Brain PET. J Nucl Med. 2005;46:1625–32. PET in suspected dementia: patterns of altered FDG metabolism. 27. Liu M, Cheng D, Yan W, Initiative ADN. Classification of Alz- Radiographics. 2014;34(3):684–701. heimer’s disease by combination of convolutional and recurrent neural networks using FDG-PET images. Front Neuroinform Publisher’s note Springer Nature remains neutral with regard to [Internet]. Frontiers; 2018 [cited 2021 Apr 6];12. Available jurisdictional claims in published maps and institutional affiliations. from: https://​www.​front​iersin.​org/​artic​les/​10.​3389/​fninf.​2018.​ 00035/​full. Accessed Jan-March 2021. 28. Shen T, Jiang J, Lu J, Wang M, Zuo C, Yu Z, et al. Predicting Alzheimer disease from mild cognitive impairment with a deep Authors and Affiliations Kobra Etminani1 · Amira Soliman1 · Anette Davidsson2 · Jose R. Chang1,3 · Begoña Martínez‑Sanchis4 · Stefan Byttner1 · Valle Camacho5 · Matteo Bauckneht6 · Roxana Stegeran7 · Marcus Ressner8 · Marc Agudelo‑Cifuentes4 · Andrea Chincarini9 · Matthias Brendel10 · Axel Rominger10,11 · Rose Bruffaerts12,13,14 · Rik Vandenberghe12,13 · Milica G. Kramberger15 · Maja Trost15,16 · Nicolas Nicastro17 · Giovanni B. Frisoni18 · Afina W. Lemstra19 · Bart N. M. van Berckel20 · Andrea Pilotto21,22 · Alessandro Padovani21 · Silvia Morbelli6,23 · Dag Aarsland24,25 · Flavio Nobili26,27 · Valentina Garibotto28 · Miguel Ochoa‑Figueroa2,7,29 Amira Soliman Nicolas Nicastro amira.soliman@hh.se Nicolas.Nicastro@hcuge.ch Anette Davidsson Giovanni B. Frisoni Anette.Davidsson@regionostergotland.se Giovanni.Frisoni@hcuge.ch Begoña Martínez‑Sanchis Afina W. Lemstra martinez_begsan@gva.es a.lemstra@amsterdamumc.nl Stefan Byttner Bart N. M. van Berckel Stefan.Byttner@hh.se b.berckel@amsterdamumc.nl Valle Camacho Andrea Pilotto MCamachom@santpau.cat pilottoandreae@gmail.com Matteo Bauckneht Alessandro Padovani matteo.bauckneht@gmail.com alessandro.padovani@unibs.it Roxana Stegeran Silvia Morbelli Roxana.Stegeran@regionostergotland.se silviadaniela.morbelli@hsanmartino.it Marcus Ressner Dag Aarsland Marcus.Ressner@regionostergotland.se daarsland@gmail.com Marc Agudelo‑Cifuentes Flavio Nobili agudelo_lau@gva.es flaviomariano.nobili@hsanmartino.it Andrea Chincarini Valentina Garibotto andrea.chincarini@ge.infn.it valentina.garibotto@gmail.com Matthias Brendel Miguel Ochoa‑Figueroa Matthias.Brendel@med.uni-muenchen.de Miguel.Ochoa.Figueroa@regionostergotland.se Axel Rominger 1 Center for Applied Intelligent Systems Research (CAISR), axel.rominger@insel.ch Halmstad University, Halmstad, Sweden Rose Bruffaerts 2 Department of Clinical Physiology, Department of Health, rose.bruffaerts@kuleuven.be Medicine and Caring Sciences, Linköping University, Rik Vandenberghe Linköping, Sweden rik.vandenberghe@uzleuven.be 3 National Cheng Kung University in Tainan, Tainan, Taiwan Milica G. Kramberger 4 Department of Nuclear Medicine, Medical Imaging Area, milica.kramberger@gmail.com Hospital Universitari i Politècnic La Fe, Valencia, Spain Maja Trost maja.trost@kclj.si 13 584 European Journal of Nuclear Medicine and Molecular Imaging (2022) 49:563–584 5 18 Servicio de Medicina Nuclear, Hospital de La Santa Creu I LANVIE (Laboratoire de Neuroimagerie du Vieillissement), Sant Pau, Universitat Autònoma de Barcelona, Barcelona, Department of Psychiatry, University Hospitals, Geneva, Spain Switzerland 6 19 Nuclear Medicine Unit, IRCCS Ospedale Policlinico San Department of Neurology, Alzheimer Center, Amsterdam, Martino, Genoa, Italy The Netherlands 7 20 Department of Diagnostic Radiology, Linköping University Department of Radiology & Nuclear Medicine, Amsterdam Hospital, Linköping, Sweden UMC, location VUmc, Amsterdam, The Netherlands 8 21 Department of Medical Physics, Linköping University Neurology Unit, Department of Clinical and Experimental Hospital, Linköping, Sweden Sciences, University of Brescia, Brescia, Italy 9 22 National Institute of Nuclear Physics (INFN), Genoa section, Parkinson’s Disease Rehabilitation Centre, FERB ONLUS – Genoa, Italy S. Isidoro Hospital, Trescore Balneario, BG, Italy 10 23 Department of Nuclear Medicine, University Hospital, LMU Department of Health Sciences, University of Genoa, Genoa, Munich, Munich, Germany Italy 11 24 Department of Nuclear Medicine, Inselspital, University Centre for Age‑Related Medicine (SESAM), Stavanger Hospital Bern, Bern, Switzerland University Hospital, Stavanger, Norway 12 25 Department of Neurosciences, Laboratory for Cognitive Department of Old Age Psychiatry, Institute of Psychiatry, Neurology, Leuven, KU, Belgium Psychology and Neuroscience, King’s College London, 13 London, UK Neurology Department, University Hospitals Leuven, 26 Leuven, Belgium Department of Neuroscience (DINOGMI), University 14 of Genoa, Genoa, Italy Biomedical Research Institute, Hasselt University, Hasselt, 27 Belgium Clinical Neurology, IRCCS Ospedale Policlinico San 15 Martino, Genoa, Italy Department of Neurology, University Medical Centre, 28 Ljubljana, Slovenia Division of Nuclear Medicine and Molecular Imaging, 16 University Hospitals of Geneva and NIMTLab, Faculty Faculty of Medicine, University of Ljubljana, Ljubljana, of Medicine, University of Geneva, Geneva, Switzerland Slovenia 29 17 Center for Medical Image Science and Visualization (CMIV), Department of Clinical Neurosciences, Geneva University Linköping University, Linköping, Sweden Hospitals, Geneva, Switzerland 13