TYPE Original Research PUBLISHED 03 November 2022 DOI 10.3389/fnagi.2022.987891 “Guttmann Cognitest”r, OPEN ACCESS preliminary validation of a digital EDITED BY Rachael D. Seidler, University of Michigan, United States solution to test cognitive REVIEWED BY Joaquin Alberto Anguera, performance University of California, San Francisco, United States Alexandru Iordan, Gabriele Cattaneo 1,2,3 *, Catherine Pachón-García 1,2,3 , University of Michigan, United States Alba Roca 1,3,4 , Vanessa Alviarez-Schulze 4,5 , Eloy Opisso 1,2,3 , *CORRESPONDENCE Gabriele Cattaneo Alberto García-Molina 1,2,3 , David Bartrés-Faz 1,4 , gcattaneo@guttmann.com Alvaro Pascual-Leone 1,6,7,8 , Josep M. Tormos-Muñoz 1,2,3 and SPECIALTY SECTION Javier Solana-Sánchez 1,2,3 This article was submitted to Neurocognitive Aging and Behavior, 1 Institut Guttmann, Institut Universitari de Neurorehabilitació adscrit a la UAB, Barcelona, Spain, a section of the journal 2 Departament de Medicina, Universitat Autònoma de Barcelona, Bellaterra, Spain, 3 Fundació Institut Frontiers in Aging Neuroscience d’Investigació en Ciències de la Salut Germans Trias i Pujol, Barcelona, Spain, 4 Departament de Medicina, Facultat de Medicina i Ciències de la Salut, Universitat de Barcelona, Barcelona, Spain, RECEIVED 18July 2022 5 Departamento de Ciencias del Comportamiento, Escuela de Psicología, Universidad ACCEPTED 10 October 2022 Metropolitana, Caracas, Venezuela, 6 Hinda and Arthur Marcus Institute for Aging Research, Hebrew PUBLISHED 03 November 2022 SeniorLife, Boston, MA, United States, 7 Deanna and Sidney Wolk Center for Memory Health, CITATION Hebrew SeniorLife, Boston, MA, United States, 8 Department of Neurology, Harvard Medical School, Cattaneo G, Pachón-García C, Roca A, Boston, MA, United States Alviarez-Schulze V, Opisso E, García-Molina A, Bartrés-Faz D, Pascual-Leone A, Tormos-Muñoz JM Thanks to technological advances, the administration of cognitive and Solana-Sánchez J (2022) “Guttmann Cognitest”r , assessments via digital solutions continues to increase, both in research preliminary validation of a digital and clinical practice. “Guttmann Cognitest”r is a digital solution for cognitive solution to test cognitive performance. Front. Aging Neurosci. 14:987891. assessment which includes seven computerized tasks designed to assess doi: 10.3389/fnagi.2022.987891 main cognitive functions requiring approximately 20 min to be completed. COPYRIGHT The purpose of the present study was to validate it against standard and © 2022 Cattaneo, Pachón-García, more extensive in-person neuropsychological assessments in the context Roca, Alviarez-Schulze, Opisso, García-Molina, Bartrés-Faz, of the Barcelona Brain Health Initiative (BBHI) cohort study. We studied Pascual-Leone, Tormos-Muñoz and 274 participants of the BBHI (126 women, mean age = 56.14, age range Solana-Sánchez. This is an 44–69), who underwent an extensive in-person assessment, including a open-access article distributed under the terms of the Creative Commons classical paper-and-pencil neuropsychological assessment and a cognitive Attribution License (CC BY). The use, assessment via the “Guttmann Cognitest”r . Principal component analysis distribution or reproduction in other indicated that “Guttmann Cognitest”r measures four main cognitive domains forums is permitted, provided the original author(s) and the copyright and convergent validity analysis demonstrated that cognitive performance owner(s) are credited and that the was associated with gold standard paper and pencil tests. Results also showed original publication in this journal is cited, in accordance with accepted an expected negative correlation with age, a relation with educational level academic practice. No use, distribution as well as a gender effect. Regression-based norming equations for the or reproduction is permitted which sample tested are also reported. Performing a cognitive assessment with this does not comply with these terms. digital solution is feasible and potentially useful to gather information about cognitive functioning in large samples and experimental settings. KEYWORDS aging, cognitive decline, cognitive functioning, computerized cognitive assessment, memory Frontiers in Aging Neuroscience 01 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 Introduction sessions to, for example, trigger alerts (Dwolatzky et al., 2003; Wild et al., 2008). Age-related cognitive decline is one of the leading public However, for people not comfortable with the use of health challenges worldwide (Minghui, 2019). In recent years, technology, computerized assessments can represent a an increasing number of studies have tried to find effective challenge, with test interfaces appearing intimidating or strategies to prevent the development of cognitive impairment, counterintuitive (Zygouris and Tsolaki, 2015), making it dementia, and functional impairment due to Alzheimer’s disease essential to implement strategies (e.g., practice trials, clear and other brain-related pathologies (Amariglio et al., 2015; instructions, etc.) that can reduce this bias, maximizing Cattaneo et al., 2018; Kulmala et al., 2018). A key need the possibility of a correct execution of the tests towards this goal is the establishment of sensitive, efficient, (Feenstra, 2018). and accessible cognitive assessments that allow the identification Another critical aspect is the need to validate these digital of preclinical stages of diseases and the detection of subtle solutions in different contexts and with different populations of cognitive changes over time. Alzheimer’s disease and other large enough sample sizes (Sternin et al., 2019). neurodegenerative diseases are preceded by a long preclinical This article aims to make a preliminary validation of phase and the possibility to detect these cases quickly and the “Guttmann Cognitest”r to explore if it can represent a accurately could potentially have important implications for self-administered, useful, and efficient instrument to measure quality of life and level of independence. For example, it will cognitive functioning in middle-aged healthy subjects for allow the implementation of preventive actions (Sternin et al., research purposes and large-scale assessments. 2019) to promote resilience to pathological processes (Pascual- Leone and Bartres-Faz, 2021). However, nowadays many of these cases remain Materials and methods undiagnosed, and the diagnosis of subtle symptoms in primary care is often difficult due to the lack of specially qualified Participants personnel, resources, or optimal tools (Mortamais et al., 2017). In this scenario, mobile technologies may offer an effective Two-hundred and seventy four participants (126 women) solution. from the Barcelona Brain Health Initiative (BBHI, Cattaneo Neuropsychological testing is a widely used and et al., 2018) took part in this study (see Table 1). standardized procedure to obtain objective indicators of Participants with a history or current diagnosis of cognitive functioning, allowing the detection of preclinical neurological or psychiatric diseases, TBI with loss of stages of dementia (Rentz et al., 2013; Duke Han et al., 2017). consciousness, substance abuse/dependence, treatment Nevertheless, classical neuropsychological testing has specific with psychopharmacological drugs, or visual impairments limitations in terms of costs and time-consumption that make it were excluded from the study. Participants provided explicit not suitable for large-scale assessments. informed consent, and the protocol was approved by the Ethics The context of the COVID-19 pandemic, moreover, showed and Clinical Research Committee of the Catalan Hospitals the need to find alternative ways to efficiently explore cognitive Union (Comité d’Ètica I Investigació Clínica de la Unió functioning, reducing people’s mobility and physical contact Catalana hospitals, CEIC18/07). between clinicians and patients. As an alternative to in-person classical neuropsychological testing, computerized assessment tools have been developed Procedures for years, and the administration of cognitive assessment via digital solutions continues to increase both in research Participants took part in a paper-and-pencil classical and clinical practice (Tierney and Lermer, 2010; De Rover neuropsychological testing session, and also a cognitive testing et al., 2011; Koo and Vizer, 2019), with several systems session via the digital solution “Guttmann Cognitest”r , in now available (O’Connell et al., 2004; Junkkila et al., 2012; the context of the in-person assessments phase of the BBHI Assmann et al., 2016). (Cattaneo et al., 2018). This kind of assessment has been shown to offer many advantages over traditional neuropsychological testing TABLE 1 Demographic characteristics of participants. (Zygouris and Tsolaki, 2015; Soldan et al., 2016), including saving costs and time. Moreover these tools could provide an Variable Mean (SD)/range Percentage (%) objective and accurate recording of responses, with enhanced Age 56.14 (6.95)/44–69 - overall sensitivity, less dependent on professional expertise or Sex - Female: 46.0 prone to human error, and with the possibility to automatically Education level - Primary: 5.1 Secondary: 27.7 store and compare a person’s performance between assessment Superiors: 67.2 Frontiers in Aging Neuroscience 02 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 These two cognitive assessments were completed in different sessions (mean delay in days = 4.60, SD = 28.20), and their order was counterbalanced across subjects. The “Guttmann Cognitest”r testing session was implemented at the Institut Guttmann facilities, partially supervised by a technician who provided the smartphone to the participant, explained the purpose of the activity, and was available for any request during the assessment. This procedure was implemented to “control” the testing setting and reduce the presence of possible bias during the validation phase. FIGURE 1 Screenshot of the video tutorial and text instructions explaining task’s logic; the user can move forward (Siguiente) and backwards (Anterior) if needed. Paper and pencil classical neuropsychological assessment and backward along it (see Figure 1). After this, the user can Two expert clinical neuropsychologists conducted the paper start the task itself. and pencil assessments (Vanessa Alviarez-Schulze and Alba When the user presses the start button, a countdown from Roca), which lasted between 60 and 90 min, with tests being 3 to 1 appears on the screen, aiming to get the user ready for administered in the same order for all participants. the task. After this, a simple demo screen of the task comes as a The test battery included well-established practice, to ensure that the user have understood the objective, neuropsychological tests, exploring main cognitive functions: rationale, and expected responses for the task. Only once the general/fluid intelligence (WAIS-IV Matrix Reasoning subtest, practice is completed correctly (there are two attempts) the task Weschler, 2008), visuospatial searching, selective attention, begins. If the practice is not completed correctly the task is not visual/motor and processing speed (Cancelation test WAIS-IV, administered, and the system moves directly to the next one, Digit symbol substitution, and Trail making test A; Reitan assuming that the person would not have been able to complete and Wolfson, 1985; Weschler, 2008), cognitive flexibility and that task correctly. After a task is completed, a final screen set-shifting (Trail making test B; Reitan and Wolfson, 1985), displays the results obtained, and the user can continue to the working memory (Digit backward, Digit forward, Letter- next task. number sequencing, Corsi Tap Test; Weschler, 2008), episodic Only the “Long-term memory” task (Task 7) does not follow memory (RAVLT; Schmidt, 1996), and visuospatial abilities this logic, as there is no tutorial nor practice screen, and the (Block design; Weschler, 2008). user is only informed that she/he will be required to remember the image-number associations seen in the previous “Short-term “Guttmann Cognitest”r digital solution memory” task (Task 2). assessment After the seven tasks, a final questionnaire is presented to gather more information about the conditions of execution, The “Guttmann Cognitest”r is a digital solution that interruptions, technical problems, or any other issues that could includes seven computerized tasks designed to assess main have affected the results of the tasks. cognitive functions from three main domains: memory, The seven tasks included in the solution are: executive functions, and visuospatial abilities. This solution is designed to be self-administered, to be potentially used for large-scale assessments and research Task 1: visual span backward purposes. After logging in, the user is presented with a welcome screen This task, designed to assess working memory, is based on containing a short description of the testing session and its the visual span backward paradigm. main purpose. Then, instructions to focus and pay attention Participants are instructed to memorize a sequence of lights on the tasks, realize the session in a quiet location, and avoid that appears sequentially, and then repeat this sequence in interruptions, are given. The time needed to complete the full reverse order. Lights appear on a grid, for a presentation time assessment is approximately 20 min. of 1 s each, and the number of elements of the series increases All tasks follow the same logic. First, an initial screen by one after a correct answer (Figure 2). In case of a wrong with a brief description of the task is presented. Then, more response, another sequence of the same length is presented. detailed instructions, together with a video tutorial, are showed, After two consecutive errors, the task ends, with the final score explaining the objective and rationale of the task. This tutorial being the number of elements of the longest sequence correctly can be repeated as needed and the user can also move forward repeated. Frontiers in Aging Neuroscience 03 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 Task 3: logic sequences Twelve logic sequences were designed to evaluate fluent intelligence and logical reasoning. Each series is composed of a 3 × 3 matrix of elements, where the element at the bottom- right is missing. The task consists of selecting one out of four possibilities, presented on the right-hand side of the screen, to complete the sequence (see Figure 4). The maximum time available to solve each series is 90 s, with an alert message FIGURE 2 appearing on the screen when there are 10 s left. Screenshot of the visual span backward task. The score for this task is the number of sequences completed correctly. Task 2: free and cued image and number Task 4: cancelation associations The symbol cancelation test was designed to assess visuo- This associative memory task consists of memorizing six spatial searching and selective attention. Participants are pairs of 2-digit numbers and images. The series of number- required to select, as fast as possible, a target of a specific shape image pairs are displayed one by one, for 2 s each, in the middle and color among a matrix of different forms. Distractors come of the screen. Once the presentation is finished, the images in the form of different shapes of the same color or the same are again presented one by one, but in a different order, and shape of a different color. The task consists of seven consecutive participants are requested to write the number associated with screens. each image (Figure 3). In the first four screens, participants must select one colored Items that are not correctly remembered are presented again shape (27 targets in total), while in the next three screens, with a cue, consisting of two possible numbers associated with they must choose two colored shapes (30 targets in total; see it, where the subject must select the correct one. The whole Figure 5). procedure is repeated three times. There are two different ways of advancing to the next screen: Then, two scores are calculated for this task: one score either the subject selects all correct answers appearing on that represents the number of correct associations reported on the screen, or the maximum 45 s threshold is reached. free recall mode, whilst the second score represents the number The total score for this task is the sum of symbols correctly of correct answers on the cued mode. selected on all screens. FIGURE 3 Screenshot of the free recall mode of the free and cued image and number associations task. Frontiers in Aging Neuroscience 04 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 The calculated score for this task is determined by the total number of correct answers and the average reaction time of those correct answers in seconds. Task 6: mental rotation This task, based on the mental rotation paradigm (Shepard and Metzler, 1971), was designed to evaluate visuospatial abilities. Participants are presented with pairs of 3D figures with different orientations, and they must indicate if they are identical FIGURE 4 Screenshot of the logic sequences task. or not, regardless of their orientation (see Figure 7), by pressing one of the two buttons shown at the bottom of the screen. The task consists of 12 couple of 3D figures, and the timeout to respond is set to 10 s. The total score of this task is the number of correct responses. Task 7: long-term memory To explore the subject’s retention capacity over time, the final task consists of presenting the same six pictures previously shown in task 2. Pictures are presented one by one, and the user must write the number associated to each picture. Similarly to FIGURE 5 the short-term task, a free recall mode comes first, whilst a cued Screenshot of the cancelation task in the two targets mode. mode is presented if the participant gives a wrong answer. The score for this task is the number of correctly remembered associations. The mean of delay between the Task 5: circle tapping immediate and delayed memory tasks (Task 2 and Task 7) was 13.48 min (SD = 4.12). This task was designed to measure visuomotor speed and sustained attention. This task shows six circles on the screen for 2 min. One by Data and statistical analysis one, each of them appears either red or blue, with a presentation time of 1 s. The participant is instructed to only press the Following the same procedure reported in previous studies screen when the circles appear in red (as fast as possible; see (e.g., España-Irla et al., 2021; Cattaneo et al., 2022; Redondo- Figure 6) and to not give any response when they appear Camós et al., 2022), we transformed raw scores obtained by in blue. classical gold-standard paper and pencil cognitive tests, and scores obtained in “Guttmann Cognitest”r , into z-scores. To create composite scores of different cognitive domains, we ran two exploratory principal component analyses (one for FIGURE 6 FIGURE 7 Screenshot of the circle tapping task. Screenshot of the mental rotation task. Frontiers in Aging Neuroscience 05 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 classical cognitive testing and another one for “Guttmann analysis. Bartlett’s test revealed a significant relationship between Cognitest”r ) using Oblimin rotation, fixing the acceptable level the factors (p < 0.001), and the Kaiser-Meyer-Olkin (KMO) test of factor loading to 0.30 (Hair et al., 1998). Based on the confirmed that the data was suitable for principal component factorial structure obtained, we calculated composite scores of analysis (KMO = 0.62). To select the number of components the different domains as the mean of the z-scores of each we used the eigenvalue-one Kaiser’s criterion (Kaiser, 1960), neuropsychological measure, multiplying it for the component controlling also the cumulative variance explained. loading value, to weigh its contribution to that component. The analysis resulted in four components that explained Moreover, we calculated a global cognition composite score, as 60.89% of the variance (see Table 2). The first component the mean of all the transformed z-scores, in line with previous included the digit span backward (0.75), digit span forward studies (Kaffashian et al., 2013; Lampit et al., 2015; Cattaneo (0.68), letter-number sequencing tests (0.68), Corsi Tap Test et al., 2022). (0.37), and WAIS-IV logical Matrices (0.59), indicating an Following this, we explored the relationship between socio- executive functions domain embracing working memory and demographic variables and cognitive results using Spearman reasoning abilities. correlations to examine the effects of age and education, and A second episodic memory domain was composed of all the using one-way ANOVA’s for biological sex. measures of the Rey Auditory Verbal Learning Test (immediate Convergent validity was assessed using Spearman’s rank recall = 0.83, delayed recall = 0.90, and recognition = 0.77). correlations between gold standard neuropsychological tests A third visuospatial searching and attentional component and results of the tasks administered with the “Guttmann comprised of the Trail Making Test A (0.72), the block design Cognitest”r in the different cognitive domains. To explore test (0.58), the digit-symbol substitution test (0.78), and the discriminant validity, we used Steiger’s Z statistic (Hittner et al., cancelation test (0.69). 2003; Diedenhofen and Musch, 2015) to compare correlation Finally, set-shifting abilities were reflected in a fourth coefficients. component, with the Trail Making Test Part B (0.81) and the Spearman’s rank correlations were used considering the no Trail Making Test Part B-A (0.87). As mentioned above, based normal distribution of almost all cognitive scores, the fact that on this factorial structure we calculated composite scores of the we did not want to make any a priori hypothesis about the four domains, and a global cognition score, as the mean of all relationship between variables (linear or monotonic), and the tests. ordinal nature of some of the variables included in the analysis. Furthermore, we estimated demographically adjusted based- “Guttmann Cognitest”r digital solution regression norms that provided the resulting z-scores metric. assessment Equations were obtained by calculating the predicted raw scores adjusted for sociodemographic variables (age, sex, and years Bartlett’s test revealed a significant relationship between the of education) that resulted in the regression model that is factors (p < 0.001), and the Kaiser-Meyer-Olkin (KMO) test statistically significant (Bezdicek et al., 2014; Cavaco et al., 2015; confirmed that the data was acceptable for principal component Kormas et al., 2018; Lavoie et al., 2018). analysis (KMO = 0.59). As in the previous analysis, the number of components was selected using Kaiser’s rule (Kaiser, 1960), checking for the cumulative variance explained. Results The analysis resulted in four components that explained 70.27% of the variance (see Table 3). A first domain was Cognitive composite score calculation and structure TABLE 2 Results of formal neuropsychological testing of participants and principal components structure. First, the results of the composite score calculation for the Cognitive domain Neuropsychological test Mean (SD) gold-standard paper and pencil neuropsychological assessment Executive fcuntions Digit span forward 6.20 (1.20) are presented, followed by the results obtained with the Digit span dackward 5.01 (1.22) same process for the “Guttmann Cognitest”r digital solution Letter-number sequencing 5.72 (1.31) Corsi tap test 6.64 (1.21) assessment. WAIS-IV logical matrices 20.09 (3.56) Episodic memory RAVLT immediate recall 55.78 (9.34) RAVLT delayed recall 12.30 (2.56) RAVLT recognizing 14.55 (1.01) Paper and pencil classical neuropsychological Visuo spatial WAIS-IV block design 46.72 (10.25) assessment searching/attention Digit symbol substitution 77.20 (13.84) WAIS-IV cancelation 41.78 (8.82) Neuropsychological raw data of the 274 participants were TMT A 26.23 (7.70) Set shifting TMT B 79.44 (28.76) transformed into z-scores and used in the principal component TMT B-A 53.20 (26.19) Frontiers in Aging Neuroscience 06 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 TABLE 3 Principal components structure for the Cognitest app. Regard education we found positive correlations with results Cognitive domain Cognitest task in memory (rs = 0.22, p < 0.001), visuo-motor speed (rs = 0.23, p < 0.001), executive functions (rs = 0.23, p < 0.001), and global Executive fcuntions and attention Logic sequences cognition (rs = 0.30, p < 0.001). Visual span backward Cancelation When we explored sex differences, similarly with what found Episodic memory Free recall previously, women outperformed males in memory (0.18 and Cued recall Delayed recall −0.15 respectively; F = 8.805, p = 0.003, η2 = 0.032), whereas Visuomotor speed Circle tapping accuracy men performed better than women in executive functions Circle tapping reaction time Visuospatial processing Mental rotation (0.28 and −0.33 respectively; F = 26.787, p < 0.001, η2 = 0.091), set shifting (0.12 and −0.15 respectively; F = 5.801, p = 0.017, η2 = 0.021), and global cognition (0.07 and −0.09 respectively; composed by logic sequences (0.71), cancelation (0.67), and F = 4.909, p = 0.028, η2 = 0.018). visual span backward (0.65) tasks, possibly reflecting attention and executive functions (attention, reasoning, and working memory). Convergent validity The second component included free recall (0.86), cued recall (0.84), and delayed recall (0.75) of images and numbers To explore convergent validity, cognitive domains calculated association tasks, indicating a potential memory component. with the results of the “Guttmann Cognitest”r digital solution A third visuomotor speed component included the circle were correlated with those obtained by paper and pencil tests tapping accuracy (0.92) and reaction times (0.93). (see Table 4). The last component comprised only the mental rotation task (0.87). Composite scores were calculated as the mean of the Memory measured by “Guttmann Cognitest”r z-scores of each task, and a global score as the mean of all tasks results. Memory correlated with results of the gold-standard paper and pencil assessment in memory (rs = 0.27, p < 0.001), visuospatial searching and attention (rs = 0.18, p = 0.004), set-shifting (rs = 0.14, p = 0.021), and global cognition (rs = 0.19, Age, sex, education, and cognition p = 0.002). When we compared correlation magnitudes (Hittner et al., Age correlated negatively with results of the “Guttmann 2003), we found that the correlation between memory measured Cognitest”r digital solution in memory (rs = −0.20, p < 0.001), by “Guttmann Cognitest”r and the same domain measured executive functions and attention (rs = −0.35, p < 0.001), by classical neuropsychological tests was higher compared visuomotor speed (rs = −0.30, p < 0.001), and mental rotation with the correlation between memory measured by “Guttmann (rs = −0.14, p = 0.027). Also, the global score in the “Guttmann Cognitest”r and set-shifting (z = 1.68, p = 0.046). However, Cognitest”r was negatively associated with age (rs = −0.48, it was similar in magnitude compared to the correlation with p < 0.001). visuo-spatial searching and attention, and global cognition We found a positive correlation between education and measured with gold-standard tests. results obtained in memory (rs = 0.28, p < 0.001), executive functions and attention (rs = −0.15, p = 0.009), and the global cognitive score (rs = 0.25, p < 0.001). Attention and executive functions measured by Regarding biological sex differences, males outperformed “Guttmann Cognitest”r women in executive functions and attention (0.19 and −0.22 respectively; F = 8.457, p = 0.001, η2 = 0.032), visuomotor Attention and executive functions measured by “Guttmann speed (0.14 and −0.17; F = 3.883, p = 0.050, η2 = 0.015), mental Cognitest”r correlated with results of gold-standard paper rotation (0.17 and −0.20; F = 8.041, p = 0.005, η2 = 0.031) and pencil neuropsychological tests in episodic memory and global cognition (0.11 and −0.12; F = 8.496, p = 0.004, (rs = 0.13, p = 0.034), visuospatial searching and attention η2 = 0.033). On the other hand, women outperformed males in (rs = 0.24, p < 0.001), executive functions (rs = 0.37, memory performance (0.98 and −0.08 respectively; F = 4.150, p < 0.001), and global cognition (rs = 0.37, p < 0.001). p = 0.043, η2 = 0.016). When we compared correlations magnitude (Hittner et al., For gold-standard and classical neuropsychological tests, age 2003) we found that the correlation between executive functions correlated negatively with visuospatial searching and attention measured by “Guttmann Cognitest”r and the same domain (rs = −0.43, p < 0.001), executive functions (rs = −0.28, measured by classical gold-standard measures was greater p < 0.001), and global cognition (rs = −0.29, p < 0.001). than the correlation between executive functions measured by Frontiers in Aging Neuroscience 07 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 TABLE 4 Correlations between results of the “Guttmann Cognitest”r digital solution and those obtained by classical paper and pencil neuropsychological testing. Gold-standard paper and pencil tests Memory Executive Visuo-spatial Set-shifting Global functions searching and attention cognition “Guttmann Cognitest”r Memory rs = 0.27** rs = 0.11 rs = 0.18* rs = 0.14* rs = 0.19* Executive functions and attention rs = 0.13* rs = 0.37** rs = 0.24** rs = 0.03 rs = 0.37** Visuo-motor speed rs = 0.04 rs = 0.09 rs = 0.26** rs = −0.22** rs = 0.06** Mental rotation rs = −0.07 rs = 0.10 rs = 0.17* rs = 0.03 rs = 0.09 Global cognition rs = 0.19* rs = 0.37** rs = 0.36** rs = 0.14* rs = 0.36** Correlation coefficients in bold represents higher correlations, at a statistically significant level. *p < 0.05, **p < 0.001. “Guttmann Cognitest”r and memory (z = 2.91, p = 0.002), the regression model (for biological sex we created a dummy and visuo-spatial searching (z = 1.78, p = 0.038). However, it variable; see Table 5 for values used), multiplied for not produced similar results in terms of magnitude in relation to the standardized “B” coefficients, model constant (k), and the Root correlation with global cognition. Mean Square Error (RMSE): Visuomotor speed measured by “Guttmann Cognitest”r z = Raw Score − [k + (Bage ∗ Age) + (Bsex ∗ sex) + (Beducation ∗ Education)] RMSE This component correlated with classical tests of visuospatial searching and attention (rs = 0.25, p < 0.001) and set-shifting abilities (rs = 0.22, p < 0.001), with similar magnitude. Task 1: visual span backward Mental rotation measured by “Guttmann Raw Score − [7.623 + (−0.060 ∗ Age) + (−0.404 ∗ sex)] Cognitest”r z = 1.361 Mental rotation results correlated only with executive functions measured with classical neuropsychological tests (rs = 0.17, p = 0.008). Task 2: free and cued images and numbers associations Global cognition measured by “Guttmann Cognitest”r Free recall The Global “Cognitest” composite score correlated with gold-standard tests of episodic memory (rs = 0.19, p = 0.003), z = Raw Score − [12.904 + (−0.136 ∗ Age) + (1.076 ∗ sex) + (1.398 ∗ Education)] visuo-spatial searching and attention (rs = 0.36, p < 0.001), 3.164 executive functions (rs = 0.37, p < 0.001), set shifting (rs = 0.14, p = 0.025), and global cognition (rs = 0.36, p < 0.001). In Cued recall relation to the magnitude of the correlation between global composite scores, it resulted in higher than the correlation between the Global “Cognitest” composite score and classical Raw Score − [16.573 + (−0.035 ∗ Age) + (0.628 ∗ sex) + (0.587 ∗ Education)] z = tests of memory (z = 2.84, p = 0.002) and set-shifting (z = 3.33, 1.707 p < 0.001). TABLE 5 Variables included in the equations and values to be used in the regression-based norms formulae. Regression-based norming equations Variables Values We estimated regression-based norms that provided Age age in years Sex Male 0 z-score metrics for each of the tasks included in the Female 1 “Guttmann Cognitest”r digital solution. Equations included Education level Primary (≤8 years) 1 Secondary (9–12 years) 2 sociodemographic variables that were statistically significant in Superiors (≥13 years) 3 Frontiers in Aging Neuroscience 08 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 Task 3: logic sequences corresponding to the cognitive domains of memory, executive functions, visuospatial searching and attention, and set-shifting. Beyond similarities, the components obtained by the two analyses did not include the same exact test, and consequently Raw Score − [12.020 + (−0.103 ∗ Age) + ( −1.010 ∗ sex) + (1.009 ∗ Education)] z = did not completely overlap in terms of cognitive processes 2.642 involved. For example, the domain of executive function for the classical neuropsychological tests includes principally working memory tests, together with a reasoning test, while the Task 4: cancelation “correspondent” component in the “Guttmann Cognitest”r includes a working memory task, a reasoning task, but also a cancelation task. Raw Score − [67.884 + (−0.246 ∗ Age) + (−0.163 ∗ sex) + (−1.240 ∗ Education)] A possible reason for this could be the greater involvement z = 11.604 of working memory, and/or inhibition processing, in the cancelation task we designed. Indeed it has been proposed that working memory could be involved in visual selective attention (De Fockert et al., 2001) and strongly related to inhibition Task 5: circle tapping (McNab et al., 2008). However, even considering these potential differences in Accuracy (%) terms of sub-processes involved in each task, we consider that, in terms of “broad” cognitive domains and functions, Raw Score − [93.934 + (−0.042 ∗ Age)] z = 1.288 the overlapping between cognitive domains measured with “Guttmann Cognitest”r and classical neuropsychological tests Reaction time (seconds) should be considerable. Raw Score − [0.343 + (−0.006 ∗ Age) + (0.032 ∗ sex)] Crucially, convergent validity analysis indicated that z = 0.099 these domains were associated with the correspondent and expected domains measured by gold-standard paper and pencil Task 6: mental rotation neuropsychological tests. In order to interpret these correspondences we specifically considered the magnitude of the effect sizes obtained in the Raw Score − [12.062 + (−0.781 ∗ sex)] z = correlation analyses, following the criteria proposed by Cohen 2.457 (1988); see also Hemphill (2003). Task 7: long-term memory For memory tasks, we found small to medium correlations with all cognitive domains, except executive functions, with the strongest of these being episodic memory, visuo-spatial searching and attention, and global cognition. Raw Score − [5.934 + (−0.018 ∗ Age) + (0.237 ∗ Education)] z = Another component measured by “Guttmann Cognitest”r , 0.680 reflecting attention and executive functions, showed medium correlations with classical tasks measuring similar Discussion attentional and executive components and the global cognition composite score, while showed only small This study aimed to make a preliminary validation of the correlations with visuo-spatial searching and memory “Guttmann Cognitest”r digital solution as a tool for measuring classically measured. cognitive functioning in a sample of healthy middle-aged adults. The visuomotor speed component correlated with tests Results showed that it is a useful and suitable instrument for this related to visuospatial searching and attention and set-shifting population. with coefficients small in magnitude. Principal component analysis on “Guttmann Cognitest”r Finally, the mental rotation component only showed a tasks showed the presence of four main components: memory, small correlation with the executive function component. executive functions and attention, visuomotor speed, and mental This could be explained considering that this test measures rotation. Moreover, it was possible to calculate a global score mainly visual ability and visual imagery (Campos, 2012) reflecting unspecific global cognitive functioning. that were not assessed in the neuropsychological testing, Similarly, also for classical neuropsychological tests, the with only a low engagement of executive functions analysis showed the presence of four main components, (Hyun and Luck, 2007). Future studies must include these Frontiers in Aging Neuroscience 09 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 tests to evaluate the convergent validity of the mental Interestingly, males outperformed women in all cognitive rotation task. domains apart from memory, where women performed better. These results are in line with previous validation studies One possible interpretation of these results could be related of computerized neuropsychological testing (see Gualtieri and to the great involvement of visuospatial components in all Johnson, 2006; Tsoy et al., 2021 for reviews) , showing great the task of the “Guttmann Cognitest”r digital solution, variability in convergent validity and concurrent validity, varying but similar results were observed also for the classical from small to large effect sizes (from 0.2 to 0.88 with an average neuropsychological tests. However, in both cases, the of 0.40). effect sizes were very small, and, in line with what was This variability makes it somehow difficult to make clear reported above, the observed differences must be interpreted a priori expectation about desirable effect sizes for a useful with caution. solution, but from our point of view coefficients lower than This is particularly important if we consider reviews and 0.2, even if statistically significant, should be interpreted with meta-analysis on this topic showing that beyond a consistently caution. observed advantage for males in mental rotation (Hyde, Considering all the obtained results, beyond differences 2014), differences in other cognitive domains are often very in magnitude between the different components, the heterogeneous (Gaillard et al., 2021), trivial in effect size (Hyde, “Guttmann Cognitest”r showed a satisfactory overall 2014), and not supported by differences in brain anatomy or convergent validity with gold-standard correspondent brain functioning (Jäncke, 2018). neuropsychological tests. To conclude, our results showed that the “Guttmann Divergent validity analysis showed a certain overlap between Cognitest”r digital solution is a suitable instrument to measure components, and almost all cognitive domains measured by cognitive functioning in different domains for research purposes “Guttmann Cognitest”r correlated with visuospatial searching and large samples assessments of middle-aged adults, supporting and attention measured by gold-standard paper and pencil tests. the potential relevance of these kinds of tools for large-scale This finding indicates that this cognitive component represents assessment. a subjacent and common cognitive process involved in all tasks, Digital solutions like this one indeed could serve as a in line with the involvement of specific cognitive processes reference point for further, more specialized, testing. Despite (e.g., working memory) in tasks designed to mainly measure their limitations, they could provide a brief picture of cognitive other cognitive functions (e.g., visual searching and attention; De functioning, helping to detect early impairments in specific Fockert et al., 2001). domains in large-scale samples. In this context, these “screening” However, the same patterns of results were found for tests should focus on functions that deteriorate first in the classical paper and pencil tests, and the relationship and overlap preclinical stage of cognitive disorders (Payton et al., 2020). between different cognitive processes involved in different tests For this reason, we decided to include very sensitive tests is a widely debated issue, exhaustively explored in the past previously related to the early detection of Alzheimer’s Disease (Vanderploeg et al., 1994; Cunningham et al., 1997; Fossati et al., (AD), like the associative memory task, sensitive to the 1999; Tremont et al., 2000; Bryson et al., 2001). medial temporal lobe dysfunctions (O’Connell et al., 2004; In clinical populations, for example, a clear dissociation De Rover et al., 2011; Junkkila et al., 2012; Soldan et al., between deficits in different cognitive domains is not always 2016). detectable, a fact that can be explained due to different levels However, this study presents several limitations that should of overlapping depending on the tests used (Fossati et al., 1999; be considered. First, due to the difference between the set Tremont et al., 2000). of tasks included in our digital solution and the classical Trying to quantify this overlap, Duff et al. (2005) found neuropsychological tests used to validate it (e.g., associative a strong relationship between different cognitive domains that memory task vs. word list), the cognitive component calculated explained up to 59% of the shared variance. may not totally overlap and reflect exactly the same subjacent For these reasons, it is sometimes appropriate to calculate cognitive processes. Second, even if we try to weigh the a global cognitive score that could be fully informative contribution to each test to a cognitive domain, each task of general cognitive functioning, and therefore, provide could contribute differently to a single domain and the different valuable information for population screening or in large domains could contribute differently to the global cognitive sample studies. score calculated. Third, the neuropsychological tests included Regarding the relation with socio-demographic variables, to validate “Guttmann Cognitest”r tasks did not include we found that age, as expected, correlated negatively with tests that mainly measure visuospatial abilities and visual performance in all cognitive domains, indicating a sort of imagery, making difficult a proper validation of the mental sensitivity of these tasks to cognitive changes due to age. On rotation task. the other hand, education correlated positively with memory, Further work is needed to examine the use of this executive functions, and global cognition. kind of cognitive assessment in clinical populations of older Frontiers in Aging Neuroscience 10 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 adults, alongside the validation of the test-retest reliability Funding and learning effect of these tests. In this regard, two versions of each task with different stimuli have already DB-F was funded by the Spanish Ministry of Science, been designed, and a specific validation study is already Innovation, Universities (RTI2018-095181-B-C21) and underway. We view this as a crucial aspect to allow us to ICREA Academia 2019 award research grants. JT-M was efficiently implement cognitive assessments and follow-ups partly supported by Fundació Joan Ribas Araquistain_Fjra, over time. AGAUR, Agència de Gestió d’Ajuts Universitaris i de Recerca (2018 PROD 00172), Fundació La Marató De TV3 (201735.10), and the European Commission (Call H2020-SC1-2016- Data availability statement 2017_RIA_777107). The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation. Acknowledgments We extend our gratitude to all participants, for their Ethics statement invaluable collaboration. No companies were involved in any step of the study, from design to data collection and analysis, The studies involving human participants were reviewed data interpretation, the writing of the article or the decision to and approved by Ethics and Clinical Research Committee of submit it for publication. the Catalan Hospitals Union—Comité d’Ètica I Investigació Clínica de la Unió Catalana hospitals, CEIC18/07. The patients/participants provided their written informed consent to Conflict of interest participate in this study. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could Author contributions be construed as a potential conflict of interest. GC, JS-S, AG-M, EO, and JT-M participated in the initial conception of the design of the “Guttmann Cognitest”r digital Publisher’s note solution. GC, JS-S, VA-S, CP-G, and AR participated actively in the data collection and analysis. AG-M, EO, DB-F, AP- All claims expressed in this article are solely those of the L, and JT-M contributed to the interpretation of the results. authors and do not necessarily represent those of their affiliated GC and CP-G drafted the article and all other authors made organizations, or those of the publisher, the editors and the critical revisions, introducing important intellectual content. reviewers. Any product that may be evaluated in this article, or All authors contributed to the article and approved the claim that may be made by its manufacturer, is not guaranteed submitted version. or endorsed by the publisher. References Amariglio, R. E., Donohue, M. C., Marshall, G. A., Rentz, D. M., Salmon, D. P., Campos, A. (2012). Measure of the ability to rotate Ferris, S. H., et al. (2015). Tracking early decline in cognitive function in older mental images. Psicothema 24, 431–434. doi: 10.1037/t144 individuals at risk for Alzheimer disease dementia. JAMA Neurol. 72, 446–454. 08-000 doi: 10.1001/jamaneurol.2014.3375 Cattaneo, G., Bartrés-faz, D., Morris, T. P., Sánchez, J. S., Macià, D., Tarrero, C., Assmann, K. E., Bailet, M., Lecoffre, A. C., Galan, P., Hercberg, S., et al. (2018). The barcelona brain health initiative: a cohort study to define Amieva, H., et al. (2016). Comparison between a self-administered and and promote determinants of brain health. Front. Aging Neurosci. 10:321. supervised version of a web-based cognitive test battery: results from the doi: 10.3389/fnagi.2018.00321 nutri net-santé cohort study. J. Med. Internet Res. 18:e68. doi: 10.2196/jmir. Cattaneo, G., Solana-Sánchez, J., Abellaneda-Pérez, K., Portellano-Ortiz, C., 4862 Delgado-Gallén, S., Alviarez Schulze, V., et al. (2022). Sense of coherence mediates Bezdicek, O., Stepankova, H., Moták, L., Axelrod, B. N., Woodard, J. L., the relationship between cognitive reserve and cognition in middle-aged adults. Preiss, M., et al. (2014). Czech version of Rey Auditory Verbal Learning test: Front. Psychol. 13:835415. doi: 10.3389/fpsyg.2022.835415 normative data. Neuropsychol. Dev. Cogn. B Aging Neuropsychol. Cogn. 21, Cavaco, S., Gonçalves, A., Pinto, C., Almeida, E., Gomes, F., Moreira, I., et al. 693–721. doi: 10.1080/13825585.2013.865699 (2015). Auditory verbal learning test in a large nonclinical portuguese population. Bryson, G., Whelahan, H. A., and Bell, M. (2001). Memory and executive Appl. Neuropsychol. Adult 22, 321–331. doi: 10.1080/23279095.2014.927767 function impairments in deficit syndrome schizophrenia. Psychiatry Res. 102, Cohen, J. (1988). Statistical Power Analysis for the Behavioral 29–37. doi: 10.1016/s0165-1781(01)00245-1 Sciences, Second Edition. New York: Routledge. Available online at: Frontiers in Aging Neuroscience 11 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 https://www.taylorfrancis.com/books/mono/10.4324/9780203771587/statistical- memory exam (GR-FNAME12) in cognitively normal elderly individuals. Clin. power-analysis-behavioral-sciences-jacob-cohen. Neuropsychol. 32, 152–163. doi: 10.1080/13854046.2018.1495270 Cunningham, J. M., Pliskin, N. H., Cassisi, J. E., Tsang, B., and Rao, S. M. (1997). Kulmala, J., Ngandu, T., and Kivipelto, M. (2018). Prevention matters: time Relationship between confabulation and measures of memory and executive for global action and effective implementation. J. Alzheimers Dis. 64, S191–S198. function. J. Clin. Exp. Neuropsychol. 19, 867–877. doi: 10.1080/01688639708403767 doi: 10.3233/JAD-179919 De Fockert, J. W., Rees, G., Frith, C. D., and Lavie, N. (2001). The role of working Lampit, A., Hallock, H., Suo, C., Naismith, S. L., and Valenzuela, M. (2015). memory in visual selective attention. Science 291, 1803–1806. doi: 10.1126/science. Cognitive training-induced short-term functional and long-term structural plastic 1056496 change is related to gains in global cognition in healthy older adults: a pilot study. Front. Aging Neurosci. 7:14. doi: 10.3389/fnagi.2015.00014 De Rover, M., Pironti, V. A., McCabe, J. A., Acosta-Cabronero, J., Arana, F. S., Morein-Zamir, S., et al. (2011). Hippocampal dysfunction in patients with Lavoie, M., Bherer, L., Joubert, S., Gagnon, J. F., Blanchet, S., Rouleau, I., et al. mild cognitive impairment: a functional neuroimaging study of a visuospatial (2018). Normative data for the rey auditory verbal learning test in the older french- paired associates learning task. Neuropsychologia 49, 2060–2070. doi: 10.1016/j. quebec population. Clin. Neuropsychol. 32, 15–28. doi: 10.1080/13854046.2018. neuropsychologia.2011.03.037 1429670 Diedenhofen, B., and Musch, J. (2015). Cocor: a comprehensive solution for the McNab, F., Leroux, G., Strand, F., Thorell, L., Bergman, S., and Klingberg, T. statistical comparison of correlations. PLoS One 10:e0121945. doi: 10.1371/journal. (2008). Common and unique components of inhibition and working memory: pone.0121945 an fMRI, within-subjects investigation. Neuropsychologia 46, 2668–2682. doi: 10.1016/j.neuropsychologia.2008.04.023 Duff, K., Schoenberg, M., Scott, J., and Adams, R. (2005). The relationship between executive functioning and verbal and visual learning and memory. Arch. Minghui, R. (2019). Risk Reduction of Cognitive Decline and Dementia: WHO Clin. Neuropsychol. 20, 111–122. doi: 10.1016/j.acn.2004.03.003 Guidelines. Geneva: Word Health Organization. Duke Han, S., Nguyen, C. P., Stricker, N. H., and Nation, D. A. (2017). Mortamais, M., Ash, J. A., Harrison, J., Kaye, J., Kramer, J., Randolph, C., et al. Detectable neuropsychological differences in early preclinical Alzheimer’s disease: (2017). Detecting cognitive changes in preclinical Alzheimer’s disease: a review of a meta-analysis. Neuropsychol. Rev. 27, 305–325. doi: 10.1007/s11065-017- its feasibility. Alzheimers Dement. 13, 468–492. doi: 10.1016/j.jalz.2016.06.2365 9345-5 O’Connell, H., Coen, R., Kidd, N., Warsi, M., Chin, A.-V., and Lawlor, B. A. Dwolatzky, T., Whitehead, V., Doniger, G. M., Simon, E. S., Schweiger, A., (2004). Early detection of Alzheimer’s disease (AD) using the CANTAB paired Jaffe, D., et al. (2003). Validity of a novel computerized cognitive battery for mild Associates Learning Test. Int. J. Geriatr. Psychiatry 19, 1207–1208. doi: 10.1002/gps. cognitive impairment. BMC Geriatr. 3:4. doi: 10.1186/1471-2318-3-4 1180 España-Irla, G., Gomes-Osman, J., Cattaneo, G., Albu, S., Cabello-Toscano, M., Pascual-Leone, A., and Bartres-Faz, D. (2021). Human brain resilience: a call to Solana-Sanchéz, J., et al. (2021). Associations between cardiorespiratory action. Ann. Neurol. 90, 336–349. doi: 10.1002/ana.26157 fitness, cardiovascular risk and cognition are mediated by structural brain Payton, N. M., Rizzuto, D., Fratiglioni, L., Kivipelto, M., Bäckman, L., and health in midlife. J. Am. Heart Assoc. 10:e020688. doi: 10.1161/JAHA.120. Laukka, E. J. (2020). Combining cognitive markers to identify individuals 020688 at increased dementia risk: influence of modifying factors and time to Feenstra, H. E. M. (2018). The Development of an Online Neuropsychological Test diagnosis. J. Int. Neuropsychol. Soc. 26, 785–797. doi: 10.1017/S13556177200 Battery: The Amsterdam Cognition Scan. Available online at: http://dare.uva.nl. 00272 Fossati, P., Amar, G., Raoux, N., Ergis, A. M., and Allilaire, J. F. (1999). Executive Redondo-Camós, M., Cattaneo, G., Perellón-Alfonso, R., Alviarez-Schulze, V., functioning and verbal memory in young patients with unipolar depression and Morris, T. P., Solana-Sanchez, J., et al. (2022). Local prefrontal cortex TMS-induced schizophrenia. Psychiatry Res. 89, 171–187. doi: 10.1016/s0165-1781(99)00110-9 reactivity is related to working memory and reasoning in middle-aged adults. Front. Psychol. 13:813444. doi: 10.3389/fpsyg.2022.813444 Gaillard, A., Fehring, D. J., and Rossell, S. L. (2021). A systematic review and meta-analysis of behavioural sex differences in executive control. Eur. J. Neurosci. Reitan, R. M., and Wolfson, D. (1985). The Halstead-Reitan Neuropsycholgical 53, 519–542. doi: 10.1111/ejn.14946 Test Battery: Therapy and Clinical Interpretation. Tucson, AZ: Neuropsychological Press. Gualtieri, C. T., and Johnson, L. G. (2006). Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs. Arch. Clin. Rentz, D. M., Parra Rodriguez, M. A., Amariglio, R., Stern, Y., Sperling, R., and Neuropsychol. 21, 623–643. doi: 10.1016/j.acn.2006.05.007 Ferris, S. (2013). Promising developments in neuropsychological approaches for the detection of preclinical Alzheimer’s disease: a selective review. Alzheimers Res. Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., and Tatham, R. L. (1998). Ther. 5:58. doi: 10.1186/alzrt222 Multivariate Data Analysis, 5th Edition. Upper Saddle River, NJ: Prentice Hall. Schmidt, M. (1996). Rey Auditory and Verbal Learning Test: A Handbook. Los Hemphill, J. F. (2003). Interpreting the magnitudes of correlation coefficients. Angeles, CA: Western Psychological Services. Am. Psychol. 58, 78–79. doi: 10.1037/0003-066x.58.1.78 Shepard, R. N., and Metzler, J. (1971). Mental rotation of three-dimensional Hittner, J. B., May, K., and Silver, N. C. (2003). A Monte Carlo evaluation objects. Science 171, 701–703. doi: 10.1126/science.171.3972.701 of tests for comparing dependent correlations. J. Gen. Psychol. 130, 149–168. doi: 10.1080/00221300309601282 Soldan, A., Pettigrew, C., Moghekar, A., and Albert, M. (2016). Computerized cognitive tests are associated with biomarkers of Alzheimer’s disease in cognitively Hyde, J. S. (2014). Gender similarities and differences. Ann. Rev. Psychol. 65, normal individuals 10 years prior. J. Int. Neuropsychol. Soc. 22, 968–977. 373–398. doi: 10.1146/annurev-psych-010213-115057 doi: 10.1017/S1355617716000722 Hyun, J. S., and Luck, S. J. (2007). Visual working memory as the substrate for Sternin, A., Burns, A., and Owen, A. M. (2019). Thirty-five years of mental rotation. Psychon. Bull. Rev. 14, 154–158. doi: 10.3758/bf03194043 computerized cognitive assessment of aging—where are we now? Diagnostics Jäncke, L. (2018). Sex/gender differences in cognition, neurophysiology and (Basel) 9:114. doi: 10.3390/diagnostics9030114 neuroanatomy. F1000Res. 7:F1000 Faculty Rev-805. doi: 10.12688/f1000research. Tierney, M. C., and Lermer, M. A. (2010). Computerized cognitive assessment 13917.1 in primary care to identify patients with suspected cognitive impairment. Junkkila, J., Oja, S., Laine, M., and Karrasch, M. (2012). Applicability of the J. Alzheimers Dis. 20, 823–832. doi: 10.3233/JAD-2010-091672 CANTAB-PAL computerized memory test in identifying amnestic mild cognitive Tremont, G., Halpert, S., Javorsky, D. J., and Stern, R. A. (2000). impairment and Alzheimer’s disease. Dement. Geriatr. Cogn. Disord. 34, 83–89. Differential impact of executive dysfunction on verbal list learning and story doi: 10.1159/000342116 recall. Clin. Neuropsychol. 14, 295–302. doi: 10.1076/1385-4046(200008)14:3;1-P; Kaffashian, S., Dugravot, A., Elbaz, A., Shipley, M. J., Sabia, S., Kivimäki, M., FT295 et al. (2013). Predicting cognitive decline: a dementia risk score vs. the Tsoy, E., Zygouris, S., and Possin, K. L. (2021). Current state of self-administered Framingham vascular risk scores. Neurology 80, 1300–1306. doi: 10.1212/WNL. brief computerized cognitive assessments for detection of cognitive disorders 0b013e31828ab370 in older adults: a systematic review. J. Prev. Alzheimers Dis. 8, 267–276. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. doi: 10.14283/jpad.2021.11 Educ. Psychol. Meas. 20, 141–151. doi: 10.1177/001316446002000116 Vanderploeg, R. D., Schinka, J. A., and Retzlaff, P. (1994). Relationships between Koo, B. M., and Vizer, L. M. (2019). Mobile technology for cognitive assessment measures of auditory verbal learning and executive functioning. J. Clin. Exp. of older adults: a scoping review. Innov. Aging 3:igy038. doi: 10.1093/geroni/igy038 Neuropsychol. 16, 243–252. doi: 10.1080/01688639408402635 Kormas, C., Megalokonomou, A., Zalonis, I., Evdokimidis, I., Kapaki, E., and Weschler, D. (2008). Wechsler Adult Intelligence Scale - Fourth Edition. San Potagas, C. (2018). Development of the greek version of the face name associative Antonio, TX: Pearson, Ed., NCS. Frontiers in Aging Neuroscience 12 frontiersin.org Cattaneo et al. 10.3389/fnagi.2022.987891 Wild, K., Howieson, D., Webbe, F., Seelye, A., and Kaye, J. (2008). The status of Zygouris, S., and Tsolaki, M. (2015). Computerized cognitive testing for computerized cognitive testing in aging: a systematic review. Alzheimers Dement. older adults: a review. Am. J. Alzheimers Dis. Other Dement. 30, 13–28. 4, 428–437. doi: 10.1016/j.jalz.2008.07.003 doi: 10.1177/1533317514522852 Frontiers in Aging Neuroscience 13 frontiersin.org