ORIGINAL RESEARCH published: 09 February 2022 doi: 10.3389/fnagi.2022.809019 Validation and Normative Data of the Spanish Version of the Rey Auditory Verbal Learning Test and Associated Long-Term Forgetting Measures in Middle-Aged Adults Vanessa Alviarez-Schulze 1,2,3,4* , Gabriele Cattaneo 1,3,5 , Catherine Pachón-García 1,2,3 , Javier Solana-Sánchez 1,3,5 , Josep M. Tormos 1,3,5 , Alvaro Pascual-Leone 1,6,7 and David Bartrés-Faz 1,8 1 Institut Guttmann, Institut Universitari de Neurorehabilitació adscrit a la UAB, Badalona, Spain, 2 Departament de Medicina, Facultat de Medicina, Universitat Autònoma de Barcelona, Bellaterra, Spain, 3 Fundació Institut d’Investigació en Ciències de la Salut Germans Trias i Pujol, Badalona, Spain, 4 Departamento de Ciencias del Comportamiento, Escuela de Psicología, Universidad Metropolitana, Caracas, Venezuela, 5 Universitat Autònoma de Barcelona, Bellaterra, Spain, 6 Hinda and Arthur Edited by: Marcus Institute for Aging Research and Deanna and Sidney Wolk Center for Memory Health, Hebrew SeniorLife, Boston, Alessandro Martorana, MA, United States, 7 Department of Neurology and Harvard Medical School, Boston, MA, United States, 8 Departament University of Rome Tor Vergata, Italy de Medicina, Facultat de Medicina i Ciències de la Salut, Institut de Neurociències, Universitat de Barcelona, Barcelona, Spain Reviewed by: Adrià Tort-Merino, Hospital Clínic de Barcelona, Spain Rey Auditory Verbal Learning Test (RAVLT) is an episodic memory helpful measure to Sara Cavaco, detect changes associated with abnormal aging. There is a lack of RAVLT validation and Centro Hospitalar Universitário do Porto, Portugal normalization studies in Spain. The aim was to determine its psychometric properties *Correspondence: and explore long-term forgetting (LTF) performance through 1-week delayed recall under Vanessa Alviarez-Schulze three different modes of administration. The RAVLT was administered to 602 cognitively valviarez@guttmann.com healthy volunteers, aged between 41 and 65 years, of whom 251 completed the LTF Specialty section: assessment. Findings reveal a factorial structure of four components, with satisfactory This article was submitted to goodness of fit, and adequate convergent and divergent validity. We also demonstrated Neurocognitive Aging and Behavior, a section of the journal the differential effect of three methodologies used in LTF assessment, supporting that Frontiers in Aging Neuroscience test expectancy positively influences long-term storage. Finally, normative data were Received: 04 November 2021 generated according to age, sex, and education. The test, including the LTF measure, Accepted: 10 January 2022 Published: 09 February 2022 is a promising tool to estimate memory in middle-aged adults and develop predictive Citation: brain aging models. Alviarez-Schulze V, Cattaneo G, Keywords: episodic memory, long-term forgetting, Memory and Learning test, validation study, Pachón-García C, Solana-Sánchez J, neuropsychological test, RAVLT validation Tormos JM, Pascual-Leone A and Bartrés-Faz D (2022) Validation and Normative Data of the Spanish Version of the Rey Auditory Verbal INTRODUCTION Learning Test and Associated Long-Term Forgetting Measures Episodic memory (EM) defined by Tulving (2002) as a neurocognitive system, particular and in Middle-Aged Adults. different from other memory domains, that enables to store specific experiences in terms of Front. Aging Neurosci. 14:809019. what happened and where and when the events happened (Pause et al., 2013) is considered doi: 10.3389/fnagi.2022.809019 a highly sensitive indicator of incipient brain pathology. Its decline has been considered as a Frontiers in Aging Neuroscience | www.frontiersin.org 1 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting predictor of cognitive impairment toward dementia, even Albu et al., 2021; Vaqué-Alcázar et al., 2021) and consists of five 10 years prior to the clinical diagnosis (Estévez-González et al., initial learning trials (Trials I–V) that imply attention, encoding, 2003; Pause et al., 2013; Boraxbekk et al., 2015; Lu et al., 2018). and reflect the ability to learn context-free auditory verbal stimuli Given its complexity, heterogeneity in EM performance has over repeated practice. After a period of 20-min, Delayed Recall been found in various clinical populations, and the definition and (Trial VI) is requested, followed by a recognition task (Trial VII), quantification of EM decline in aging remain a questioned topic allowing to assess evocation and storage processes (Schoenberg (Pause et al., 2013; Nyberg and Pudas, 2019). For this reason, the et al., 2006; Lezak et al., 2012; Cavaco et al., 2015; Puerta Lopera existence of adequately validated and standardized psychometric et al., 2018). On the other hand, other versions also contain tests for memory measurement in different modalities (verbal and an interference list of 15 words (list B) after trials 1–5, which visual) is essential to identify healthy people, higher performers, measures the interference effect (Schmidt, 1996; Lezak et al., or maintainers (successful memory aging), as well as those 2012). subjects with memory decline who show a negative rate of change The classic RAVLT quantification, the total number of words in longitudinal measurements (Nyberg and Pudas, 2019). recalled in each trial, has been shown to reflect specific patterns Within the mnesic function, LTF is a dimension that has in healthy and clinical populations (Libon et al., 2015; Lu been less studied (Baddeley et al., 2019). Accelerated long- et al., 2018). However, some authors highlight the relevance term forgetting (ALF) is defined as a phenomenon by which of taking executions errors (perseverations and intrusions) into memories that are encoded and maintained during intervals of account when analyzing memory tasks because they are useful to approximately 30 min are then forgotten faster than expected characterize abnormal brain aging (Bezdicek et al., 2014; Ferreira over delays of days/weeks (Elliott et al., 2014). However, this Correia and Campagna Osorio, 2014; Libon et al., 2015; Montero definition is currently under review because there are no unique and Cáceres, 2017; Weitzner et al., 2020). methods for calculating the ALF score (Butler et al., 2019). The RAVLT has been previously standardized and validated in The study of ALF shows high potential to improve Alzheimer’s different languages, with different health and clinic populations Disease (AD) prediction, and it has been strongly recommended (Schoenberg et al., 2006; Messinis et al., 2007; Fichman et al., to include it within longitudinal research (Tort-Merino et al., 2010; Vakil et al., 2010; Bezdicek et al., 2014; Speer et al., 2021a). Recent findings have suggested that LTF measures are 2014; Cavaco et al., 2015; Lavoie et al., 2018). Although there much more sensitive to assess earlier pathological memory are studies with Hispanic populations (Marqués et al., 2013; decline associated with mild cognitive impairment (MCI) and Ferreira Correia and Campagna Osorio, 2014; Sánchez-Nieto AD (Geurts et al., 2015). In fact, Wearn et al. (2020) found et al., 2016; Puerta Lopera et al., 2018), there have been that the long-term delay period can improve the detection of no published validation and standardization studies in recent subjects likely to decline during the following year. Also, there decades with a representative healthy middle-aged Spanish is evidence of the relation between ALF and early markers of sample, and its adapted and validated versions do not formally AD, including subjective complaints, autosomal dominant AD include LTF measurement. mutation carriers, APOE E4 carriers, and abnormal levels of CSF Moreover, even if LTF of this test has been previously used Aβ42 and CSF Aβ42/ptau ratio (Tort-Merino et al., 2017, 2021a,b; in experimental studies, the procedures employed were not Reiman, 2018; Weston et al., 2018; Zimmermann and Butler, homogeneous (Mameniskiene et al., 2006; Butler et al., 2007; 2018; Butler et al., 2019). Atherton et al., 2019; Savage et al., 2019). In this vein, Elliott There is an urgent need for reliable, valid, and standardized et al. (2014) introduced some methodological issues regarding LTF measures (Elliott et al., 2014; Baddeley et al., 2019; Mayes LTF assessment. The main problem is the need to eliminate et al., 2019), considering that most of the long-term episodic or minimize the possibility of rehearsal during delays. To solve memory tests usually limit retrieval to 20/30 min. this, Elliott et al. (2014) mentioned that some researchers have The RAVLT is a widely used episodic verbal memory test decided not to forewarn participants about a deferred evocation that measures encoding, consolidation, storing, and retrieval of requirement (Weston et al., 2018; Wearn et al., 2020; Tort- verbally learned content (Schmidt, 1996; Schoenberg et al., 2006). Merino et al., 2021a). However, this method could not be the Previous research suggests that the RAVLT is helpful to predict most appropriate within the clinical practice or longitudinal the progress of MCI to dementia (Estévez-González et al., 2003; studies. If later assessments are needed, the comparisons between Schoenberg et al., 2006; Drolet et al., 2014; Marchand et al., 2017; longitudinal points in time would be biased due to the effect of the Moradi et al., 2017). test expectation on the consolidation process. Test expectation The RAVLT is a 15-unrelated-word verbal list-learning task refers to the assumption of the “future relevance” of learned (Rey, 1964; Schmidt, 1996). Different versions of this instrument information given the warning of a long-term delayed recall have been developed, with variations in its application procedures (Shimizu, 1996; Wamsley et al., 2016). (Estévez-González et al., 2003; Lezak et al., 2012; Bezdicek et al., Alternatively, other authors assumed a procedure that consists 2014; Cavaco et al., 2015) that need to be considered for the of asking the participants intentionally not to rehearse but comparison between studies, particularly for normative data explicitly did not forewarn them about the delayed retrieval selection (Messinis et al., 2007). after days/weeks; they were only informed that they would The version that we aim to validate is frequently used in receive a call to complement the evaluation (Butler et al., 2007; Spain for clinical practice and research (Perea Bartolomé et al., Muhlert et al., 2010; Hoefeijzers et al., 2013; Savage et al., 2019). 2000; Estévez-González et al., 2003; García-Rudolph et al., 2020; Nevertheless, as in the procedure described above, subjects may Frontiers in Aging Neuroscience | www.frontiersin.org 2 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting predict that they will undergo an LTF testing again in longitudinal psychopharmacological drugs (n = 9) were excluded from the studies or clinical follow-up assessments. study. Also, we did not include the participants with objective In some other cases, the procedure described by the deficits in neuropsychological tests (n = 15) (see the Section researchers is not clear (Davidson et al., 2007; Atherton et al., “Procedures and Materials”) and those whose assessment was 2019). Considering that in follow-up measures the subjects could significantly interfered with (n = 4) for a variety of reasons predict that they will be asked for a delayed recall, another (external interruptions, prior familiarity with the test, and task possible application modality would be to inform them about abandonment). The participants provided explicit informed the LTF probe requiring them not to practice. However, we consent, and the protocol was approved by the Comitéd’ hypothesize that test expectations and knowing that learned Èticai Investigació Clínica de la Unió Catalanad’ Hospitals content will be asked could affect LTF performance. (Cattaneo et al., 2018). It is essential to discuss these methodological concerns because there is evidence that rehearsal may decrease LTF (Elliott Procedures and Materials et al., 2014). Still, the potential effect of the different application The RAVLT was administered according to the standardized modalities described above is unknown. Thus, possibly each procedure during the BBHI cognitive assessment session application version would require specific normative data. (Cattaneo et al., 2018). Its administration lasted 30–35 min, In conclusion, given the importance of assessing episodic including the lapse required to measure delayed recall. It is crucial memory as a preclinical indicator of abnormal aging, our to highlight that no other memory tests were applied between the main objectives were: (1) to determine the RAVLT validity and RAVLT trials. The version used consists of the oral presentation develop population-specific normative data in a Spanish sample of a list of 15 unrelated Spanish words (Perea Bartolomé et al., of healthy individuals aged between 41 and 65 years; (2) to 2000; Estévez-González et al., 2003). The application procedure explore sample performance on the proposed LTF measure with started with the initial learning (the encoding phase) of a 15- a 1-week delayed recall using RAVLT under three different word list, which is read by the investigator at the rate of one administration modalities. word per second, followed by an immediate recall. The examinees should repeat all the words they could remember, regardless of the order. This operation was continuously repeated five times (I– MATERIALS AND METHODS V trial). After 20–25 min, the participants were then requested to retrieve as many list words (delayed recall). Finally, a recognition Participants task (recognition) required the examinees to identify the words This study was conducted using data collected in 2019 practiced within a broader list that includes new content. on a subset of the participants enrolled in the in-person During the BBHI cognitive assessment session (Cattaneo assessment of the Barcelona Brain Health Initiative (BBHI), et al., 2018), a comprehensive neuropsychological tests battery a longitudinal prospective population-based cohort study was administered in the following fixed order: S-FNAME Exam (Cattaneo et al., 2018). (Alegret et al., 2015; Alviarez-Schulze et al., 2022) direct and inverse digit spans (Peña-Casanova et al., 2012), Trail Making Validation and Normative Sample Test parts A and B (TMT-A and TMT-B, respectively) (Peña- The validation sample consisted of 602 volunteers (280 women, Casanova et al., 2012), phonemic and semantic fluency tasks mean age = 53.50, SD = 6.96, age range, 41–65, mean years (Peña-Casanova et al., 2012), Matrix Reasoning subtest from of education = 17.21, SD = 3.74, 95% Caucasians) who had Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) completed the entire neurocognitive assessment protocol at the (Wechsler, 2012), Rey Auditory-Verbal Learning Test (RAVLT; moment of the analysis. Most of the participants were Catalonia Schmidt, 1996), Block Design subtest from WAIS-IV (Wechsler, residents (96.01%) and only 3.99% came from other areas within 2012), Letter-Number Sequencing (Peña-Casanova et al., 2012), Spain. About 95.01% of our participants were Catalan-Spanish Digit-Symbol Substitution Test and Cancelation subtests from bilinguals (4.99% were only Spanish speakers). WAIS-IV, and Corsi block-tapping test (Peña-Casanova et al., 2012). The cognitive assessment session was conducted by two Long-Term Forgetting Sample expert neuropsychologists and lasted approximately 90 min. Among the total participants who completed the in-person At the end of the in-person testing, the participants received neuropsychological assessment, 251 subjects also completed one of the three different instructions about the 1-week call, as we the LTF assessment with a 1-week delayed recall. The rest mentioned above, to measure LTF through the RAVLT word list of the sample could not be contacted by telephone in the free recall and recognition task. Specifically, the LTF procedure scheduled period. These participants were pseudo-randomly modalities were: assigned to one of three groups to assess and compare three different administration procedures. Groups were balanced for • The participants of Group I (n = 83) were warned that sex, age, and education. they would receive a phone call to collect some general Following BBHI exclusion criteria, the participants with information, without any reference that what will be a history or current diagnosis of neurological or psychiatric requested is related to neuropsychological evaluation. disease (n = 13), TBI with loss of consciousness (n = 4), • The participants of Group II (n = 85) were warned substance abuse/dependence (n = 9), or treatment with about a call to answer some questions related to the Frontiers in Aging Neuroscience | www.frontiersin.org 3 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting neuropsychological assessment conducted during the in- verify the fit of the factor structures obtained from the EFA person session. Still, they were not specifically informed to choose the most plausible model avoiding indeterminacy about the LTF measure, and, however, they were explicitly bias, following the rival model strategy proposed by Hair asked not to rehearse material or any of the activities et al. (1999). To examine, the goodness of fit considered the included in the neuropsychological assessment. absolute, incremental, and parsimonious fit indices: Chi-square • The participants of Group III (n = 83) were explicitly (χ2 ), Normed Chi-square (χ2 /df), Goodness of Fit Index (GFI), forewarned about the 1-week delayed retrieval of the Adjusted Goodness Fit Index (AGFI), Root Mean Square Error RAVLT word list, and they were directly and precisely of Approximation (RMSEA), Normed Fit Index (NFI), Tucker- requested not to practice the word list. Lewis Index (TLI), parsimonious normed fit index (PNFI), and Parsimony Goodness-of-Fit Index (PGFI) (Hair et al., 1999). Previous LTF research (Weston et al., 2018; Zimmermann Convergent validity was calculated using Pearson correlation and Butler, 2018) has fixed a criterion of the accuracy of 80% coefficients between RAVLT scores (Immediate Total Recall and of acquired information during initial learning. However, in this Delayed Recall) and S-FNAME scores (subtotal scores of face- study, we followed one of the standardized versions of the RAVLT name and face-occupation association: S-FNAME FN-N and (Perea Bartolomé et al., 2000; Estévez-González et al., 2003) that S-FNAME FN-O). Divergent validity was assessed using Pearson consists of five initial learning trials, considering that we aimed correlation coefficients with non-memory tests: TMT-A, TMT-B, to validate the test and generate its normative tables for the Matrix Reasoning, and Block Design subtests. Spanish population. To determine if the participants did rehearse after the testing session, we asked during the phone call if they wrote down Normative Data Analysis the information, the words were spontaneously evoked, or The correlation between demographic variables and RAVLT voluntarily practiced during the week. We excluded those who scores was calculated using Pearson correlation coefficients. We answered affirmatively. Only one subject of Group II reported carried out multiple linear regression analyses to examine the having practiced, and four participants of Group III were contribution of sex, age, and education on RAVLT performance. excluded for this reason. Analysis of variance (ANOVA) was used to determine the effect of sociodemographic variables, taking into account those Data Analysis regression models that explained at least 5% of the total Statistical analyses were executed using SPSS version variance of RAVLT scores (Peña-Casanova et al., 2012; Ferreira 22.0 (Statistical Package for Social Sciences, Chicago, IL, Correia and Campagna Osorio, 2014). Therefore, ANOVA was United States). Statistical significance was determined when calculated to determine the effect of sex, age ranges (41– p < 0.05 for all the inferential analyses. 48, 49–57, ≥58 years), and education level (<16 years of education ≥ 16 years of education) on RAVLT measures. Validation Data Analysis The configuration of the age groups was data driven and The distribution of sociodemographic variables (age ranges, corresponds to the terciles of the sample to guarantee a similar educational level, and sex) of the validation sample was sample size within the bands. Specifically, it was obtained after estimated. Years of education were measured by explicitly multiple comparisons between different ranges to guarantee that asking the volunteers to inform the total time of formal the resulting groups reflected significant differences in RAVLT education, counting from the beginning of obligatory scores, instead of arbitrarily dividing age (Ferreira Correia education in Spain (primary school). Spanish educational and Campagna Osorio, 2014; Alviarez-Schulze et al., 2022). system includes elementary/primary school (6 years), obligatory The division of the groups by years of education corresponds secondary school (4 years), and baccalaureate/high school to the level of higher education (16 years), considering the or middle-grade vocational training (2 years). Higher distribution of our sample. education comprises undergraduate degrees (4 years) Finally, the RAVLT norms were developed following the and post-graduate degrees (specialization, master’s, and traditional norming method, considering the combination of Ph.D. programs). those demographic variables that demonstrated a significant Descriptive analyses were performed for RAVLT trials and effect. Additionally, we estimated demographically adjusted errors (repeated words, intrusions, and repeated intrusions). We based-regression norms that provide the resulting z-scores metric included complementary scores: total recall (sum of trials I, II, III, obtained through the statistical procedure described below IV, and V) and the learning over trial (LOT) score corresponding (Bezdicek et al., 2014; Cavaco et al., 2015; Kormas et al., 2018; to total learning corrected for an immediate word span [Total Lavoie et al., 2018): Recall–(5 × Trial I)]. The latter estimates better the improvement across trials than other scores (Vakil et al., 2010; Marqués et al., 1. From the regression equation obtained by multiple 2013; Cavaco et al., 2015). Also, we calculated the forgetting rate regression analyses, we calculated the predicted raw scores [1 - (delayed recall/Trial V) and multiplied by 100]. adjusted for sociodemographic variables (age, sex, and We ran an exploratory factor analysis (EFA) using principal years of education) that resulted statistically significant. component analysis and OBLIMIN rotation. Confirmatory factor A case wise diagnosis identified possible outliers, and we analysis (CFA) was carried out using IBM SPSS AMOS to confirmed the regression assumptions were met. Frontiers in Aging Neuroscience | www.frontiersin.org 4 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting 2. The residuals were calculated by subtracting the predicted TABLE 1 | Frequencies of demographic characteristics of a validation sample. value from the observed raw score. Age 41–49 years 50–57 years 58–65 years Total 3. Finally, we standardized the residuals by dividing them by range the standard error of the estimate (SEE) of the regression line. The z-scores obtained are interpreted using a Z n = 204 n = 195 n = 203 distribution table to determine the examinees’ performance compared to their normative group. n % n % n % n % Years of 0–15 48 23.5 53 27.2 80 39.4 181 30.1 Long-Term Forgetting Data Analysis Education The distribution of sociodemographic characteristics of the LTF ≥16 156 76.5 142 72.8 123 60.6 421 69.9 sample according to sex, age ranges (41–49, 50–57, 58–65 years), Gender Women 94 46.1 93 47.7 93 45.8 280 46.5 and educational level (<16 and ≥16 years) was calculated for each Men 110 53.9 102 52.3 110 54.2 322 53.5 group under the different administration modalities proposed. Descriptive analyses for each group were performed for 1- TABLE 2 | Description of RAVLT, errors, and complementary scores. week delayed measures: 1-week Delayed Recall, 1-week Delayed Recognition, 1-week intrusions, and 1-week Forgetting Rate. Min Max Mean SD One-week Forgetting Rate refers to the ratio of information loss I 2 13 6.26 1.77 between the 25 min Delayed Recall and 1-week Delayed Recall II 5 15 9.62 2.19 scores. It was calculated using the formula: [1 - (1-week Delayed III 5 15 11.27 2.09 Recall/25-min Delayed Recall)] × 100. IV 6 15 12.21 1.90 We ran a one-way ANOVA to compare group performance on V 6 15 12.73 1.87 the cognitive tests administered during the neuropsychological Immediate Total Recall 29 71 52.09 8.19 assessment session, including the RAVLT measures. These Delayed Recall 3 15 11.36 2.56 analyses were performed to ensure homogeneity between groups. Recognition 8 15 14.41 1.06 A mixed ANOVA was conducted to determine whether Repeated words 0 18 4.85 3.59 changes in the number of words evoked are a result of the Intrusions words 0 6 0.67 0.94 interaction between the administration modality and the time Repeated intrusions 0 8 0.33 0.92 lapse of delayed recall. This analysis will determine changes LOT 3 40 20.81 6.45 between the 25-min and 1-week delayed recall measures that Forgetting Rate −25 62.5 11.09 13.96 depend on the application modality corresponding to the groups I, Trial I of Rey Auditory Verbal Learning Test (RAVLT); II, Trial II of RAVLT; III, Trial (Interaction Effect). III of RAVLT; IV, Trial IV of RAVLT; V, Trial V of RAVLT. LOT, Learning Over Trial or Finally, we ran linear regression analyses for each group to Immediate Total Recall – (5 × Trial I). explore the possible influence of sociodemographic variables on the LTF measures (1-week Delayed Recall, 1-week Recognition, Construct Validity and 1-week Forgetting Rate). Exploratory factor analysis (EFA) was carried out using principal component analysis and Oblimin rotation (Hair et al., 1999). The Kaiser–Meyer–Olkin measure of sampling RESULTS adequacy (KMO = 0.868) and Bartlett’s test of sphericity (χ2 = 2922.23; gl = 45; p < 0.001) were satisfactory, and Rey Auditory Verbal Learning Test determinant of correlation matrix 0.007 tended to 0 as expected Validation (Hair et al., 1999). The distribution of sociodemographic variables (age ranges, The EFA yielded three factors with an eigenvalue greater than education, and sex) of the validation sample is presented in 1.00. This solution explained 70.15% of the total variance. The Table 1. Distribution by sex was homogeneous according to 3-factor model showed that Factor 1 loads Trials I, II, III, IV, a one-sample binomial test (p = 0.095). Also, the number of and V, Delayed Recall and Recognition; Factor 2 was related to males and females within each age ranges [χ2 (2) = 0.165; Intrusions and Repeated intrusions words; and Factor 3 only p = 0.921] was uniform. loads repeated words, as presented in Table 3. An unequal sample distribution (p < 0.001) by educational Considering the criteria of the Scree plot and the explained level (<16 years of education ≥ 16 years of education) was variance above 60%, a 2-factor model was found. One factor observed. Also, the distribution of years of education was is related to RAVLT Trials, and the other includes errors (only unbalanced between age bands, with a larger proportion of Intrusions, repeated words did not load on any factor). education below 16 years in the older individuals. However, the Finally, before carrying out CFA, we established an a priori educational level was similar between males and females [χ2 factorial structure from a theoretical basis of the construct and (1) = 0.74; p = 0.39]. previous findings (Vakil and Blachstein, 1993; Baños et al., 2005) Descriptive analysis for RAVLT trials, errors, and that suggest one component associated with the Attention and complementary scores (Immediate Total Recall, LOT, and Memory span (Trials I and II) and others related to Memory Forgetting Rate) was carried out (see Table 2). and Learning (Trials III, IV, V, Delayed Recall, and Recognition); Frontiers in Aging Neuroscience | www.frontiersin.org 5 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting TABLE 3 | Rey Auditory Verbal Learning Test factor structure obtained from EFA. TABLE 4 | Correlation coefficients between RAVLT scores, memory and non-memory tests scores. Rotated components matrix S-FNAME TMT-A TMT-B Reasoning Block Design Component Matrix 1 2 3 Immediate Total 0.43** −0.21** −0.22** 0.21** 0.17** Recall IV 0.87 −0.19 −0.03 Delayed Recall 0.37** −0.19** −0.18** 0.20** 0.14** III 0.87 −0.21 −0.01 ** p < 0.01. Delayed Recall 0.85 −0.24 −0.13 S-FNAME, Spanish FNAME Exam total score; TMT-A, Trail Making Test Part A; V 0.84 −0.17 −0.08 TMT-B, Trail Making Test Part B. II 0.84 −0.12 0.18 I 0.65 −0.07 0.24 Recognition 0.62 −0.18 −0.09 measures. We found positive associations (p < 0.01) between Repeated Intrusions −0.14 0.88 0.02 RAVLT scores and TMT-A, TMT-B, Matrix Reasoning and Intrusions words −0.24 0.87 0.09 Block Design (WAIS IV), but with small effect size and Repeated words 0.00 0.05 0.95 negative correlation (p < 0.01) with TMT-A and TMT-B (time measurements), with a small effect size as well (see Table 4). I, Trial I of Rey Auditory Verbal Learning Test (RAVLT); II, Trial II of RAVLT; III, Trial III To confirm evidence of divergent validity, we used of RAVLT; IV, Trial IV of RAVLT; V, Trial V of RAVLT. Steiger’s z statistic to compare correlations between RAVLT scores and non-memory tests scores with those obtained by we included a third dimension composed of Intrusions and a measuring convergent validity. Correlation between the RAVLT fourth component related to repeated words. Therefore, a four- performance (Immediate Total Recall and Delayed Recall) and factor model resulted. TMT scores was lower than the correlation between the RAVLT Confirmatory factor analysis results reflected comparisons and both S-FNAME scores (Steiger’s z = −4.46, p < 0.001, between absolute fit indicators, incremental fit measures, and Steiger’s z = −3.57, p < 0.001; Steiger’s z = −4.37, p < 0.001; parsimony of each rival model (see Supplementary Table 1 Steiger’s z = −3.85, p < 0.001). Likewise, Matrix Reasoning and for details). All chi-squares reflected high values, contrary to Block Design reflected lower correlation coefficients with the expectations, but this indicator is not sensitive in the case of RAVLT performance than the coefficients between the RAVLT large samples. χ2 /gl index seems less sensitive to the sample and both S-FNAME scores (Steiger’s z = −4.29, p < 0.001; size; smaller magnitudes are considered a better fit. The 4-factor Steiger’s z = −3.39, p < 0.001; Steiger’s z = −5.29, p < 0.001; model showed lower χ2 /gl, below 5, a cut-off point indicating an Steiger’s z = −4.58 p < 0.001). acceptable level (Hair et al., 1999). All GFI indices were adequate (>0.9), although the 4-factor Normative Data model reflected superior fit (GFI = 0.95). Concerning RMSEA, To explore possible associations between demographic variables one of the most critical indicators, the 4-factor model was the and the RAVLT measures, we calculated Pearson correlation only one that fell within the acceptable range below 0.08 (Hair coefficients. Negative correlations between age and RAVLT et al., 1999; Batista and Coenders, 2000). scores and positive association between years of education and Regarding the incremental fit indicators, the 4-factor model test performance were found (p < 0.01) (see Supplementary showed an adequate value, near to 1; the other models obtained Table 2 for details). unsatisfactory values below 0.90. NFI and NNFI for all models Thus, we ran multiple regression analyses to confirm the reflected deficient values. Finally, parsimonious fit indices contribution of sex, age, and years of education as predictors (PGFI and PNFI) of all models reflected unsatisfactory values. of RAVLT scores (see Table 5). Findings revealed a significant Therefore, these indices are not useful as a criterion to compare contribution of these variables to the variance of RAVLT and complement the choice of the best fit model. Immediate Total Recall (p < 0.001) and Delayed Recall The most important and relevant index to select (p < 0.001). Nonetheless, the regression model for Recognition the best model is the absolute fit indices, especially measure only explained 4.1% and for Forgetting Rate only RMSEA. Therefore, the 4-factor model, which is consistent explained 2%. Additionally, regarding error measures, a very low with previous findings, is chosen to explain the factor contribution to the total variance (2%) of Repeated words (the structure of the RAVLT. model explained p < 0.01) and regression models for Intrusions (p = 0.14) and Repeated Intrusions (p = 0.07) were not significant. Convergent and Divergent Validity The effects of age (divided into ranges: 41–49, 50–57, To examine the convergent validity of the RAVLT, Pearson and 58–65 years), educational level (<16 and ≥16 years correlation coefficients (r) were calculated between RAVLT of education), and sex were calculated using ANOVA with scores and the S-FNAME performance. Statistically significant Bonferroni correction since the condition of equality of variances (p < 0.01) associations were found with medium effect size. was demonstrated using Levene test. In addition, regarding divergent validity, we obtained Pearson Age [F(2,590) = 12.26, p < 0.001, η2 = 0.04], sex correlation coefficients between RAVLT scores and non-memory [F(1,590) = 21.10, p < 0.001, η2 = 0.04], and education Frontiers in Aging Neuroscience | www.frontiersin.org 6 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting TABLE 5 | Contribution of age, gender, and education on RAVLT scores. TABLE 6 | Normative data for women performance on RAVLT Immediate Total Recall. Measure Predictor Standardized β T p value R2 Women Immediate Total Age −0.24 −6.31 <0.001 0.13 Recall 41–49 years 50–57 years 58–65 years Gender −0.22 −5.65 <0.001 Education (years) 0.14 3.54 <0.001 Percentile <16 yrs ≥16 yrs <16 yrs ≥16 yrs <16 yrs ≥16 yrs educ educ educ educ educ educ Delayed Recall Age −0.19 −4.84 <0.001 0.09 Gender −0.19 −3.70 <0.001 2 37 44 35–36 33–37 29–32 40 Education (years) 0.15 4.17 <0.001 5 38–41 45–46 37–41 38–43 33–39 41 Recognition Age −0.10 −2.47 0.01 0.04 10 42–44 47–49 42–43 44–45 40–41 42–43 Gender −0.0.9 −2.15 0.03 15 45 – 44 46 42–43 44–45 Education (years) 0.15 3.66 <0.001 20 46 50 45–46 47–48 44–47 – Forgetting Rate Age 0.10 −2.54 0.01 0.02 25 47 51–52 47–48 49 – 46–47 Repeated Age 0.05 1.21 0.23 30 48 53 49 50 48 48 words 35 49 54–55 50 51 – 49 Gender −0.14 −3.33 <0.001 0.02 40 – 56 51 52 49 50–51 Education (years) 0.02 0.51 0.61 45 50 57 52–53 – 52 50 51 58 54 53–54 50–53 53 55 52–53 59 – 55 54 54 [F(1,590) = 11.38, p < 0.001, η2 = 0.02] showed significant impact 60 54 – 55 56 55 55 on Immediate Total Recall. Post hoc analyses revealed differences 65 55–56 60 56 57–58 – 56 between all age groups (p < 0.001), reflecting that as the age 70 57 – – 59 56 – range increases, the performance decreases significantly. Men, 75 58–59 62–63 57 60 – 57–58 older, and less educated individuals obtained a lower Immediate 80 – 64 58–59 61–62 57–58 59 Total Recall score. 85 60 65 60 63 59–61 60–62 Regarding Delayed Recall, sex [F(1,590) = 18.71, p < 0.001, 90 61–62 66 61–65 64 62–64 63–67 η2 = 0.03], age [F(2,590) = 7.29, p < 0.001, η2 = 0.02], 95 63 67–68 66–67 65–67 65 68–69 and education [F(1,590) = 8.82, p < 0.01, η2 = 0.02] groups 98 – ≥ 69 – ≥68 – ≥ 70 differed significantly in terms of their performance, reflecting <16 years of education, less than 16 years of education; ≥16 years of education, the same pattern described for Immediate Total Recall scores. 16 years of education or more. Post hoc analyses showed statistically significant difference between the youngest (41–48 years old) and oldest (58–65 years years according to the regression models for Immediate Total old) groups (p = 0.003) and between the middle-aged range Recall and Delayed Recall (see details in Supplementary Table 4). (49–57 years old) and the oldest group. Interaction effects between sociodemographic variables were not found for any of the RAVLT measures. Long-Term Forgetting Assessment Then, we stratified and described Immediate Total Recall Sex, age ranges (41–49, 50–57, and 58–65 years), and educational and Delayed Recall scores by sex, age, and educational level (<16 and ≥16 years) are represented in Table 11. We level according to ANOVA results (see Supplementary found the distributions by sex [χ2 (2) = 0.29; p = 0.98], age Table 3) to generate traditional population-specific norms range [χ2 (4) = 5.82; p = 0.21], and educational level [χ2 of healthy Spanish individuals aged between 41 and 65 (see (2) = 0.254; p = 0.88] were similar between the three groups. Thus, Tables 6–9). Stratified percentile tables were not generated for they are homogeneous according to their sociodemographic Recognition, Forgetting Rate, and Repeated word measures characteristics. since the regression models explained less than 5% of the We found no differences between the performance of the total variance. Likewise, the regression models for other groups in any neuropsychological measure according to one-way error scores were not statistically significant; therefore, ANOVA tests calculated. It is important to note that they did normative tables were developed for the entire sample not differ in the RAVLT scores (see Supplementary Table 5 for distribution (Table 10). details). Therefore, they are homogeneous groups in terms of It is important to note that the 15th percentile their cognitive execution, especially in their performance on the corresponds to mild impairment (a score more than one RAVLT Immediate Total Recall, Delayed Recall, and Recognition. SD below the mean), and the 2nd percentile is the cut-off Descriptives of the performance of the groups on the point for a significantly impaired performance (two SDs LTF measures (1-week Delayed Recall, 1-week Recognition, below the mean). Intrusions, and 1-week Forgetting Rate) are presented in Additionally, we estimated demographically adjusted Table 12. equations to calculate RAVLT z scores through the regression- Regardless of application modality, the participants evoked based norming method. We adjusted scores for age, sex, and fewer words from the list in the 1-week delayed recall than the Frontiers in Aging Neuroscience | www.frontiersin.org 7 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting TABLE 7 | Normative data for men performance on RAVLT Immediate Total Recall. TABLE 9 | Normative data for men performance on RAVLT Delayed Recall. Men Men 41–49 years 50–57 years 58–65 years 41–48 years 49–57 years 58–65 years Percentile <16 yrs ≥16 yrs <16 yrs ≥16 yrs <16 yrs ≥16 yrs Percentile <16 yrs ≥16 yrs <16 yrs ≥16 yrs <16 yrs ≥16 yrs educ educ educ educ educ educ educ educ educ educ educ educ 2 39 35–38 – 33–36 33 33–35 2 6 5–6 5 5 – 4 5 40–41 39–42 37–39 37–39 34–36 36 5 7 7 6 6 6 5–6 10 42–44 43–46 40 40–41 37–38 37–38 10 8 8–9 7 7–8 7 7 15 45 47 41 42 39 39–40 20 9 10 8 9 – 8 20 46–48 48 42 43–44 40–41 41 30 – 11 – 10 8 9 25 – 49 – 45–46 – 42–43 40 10 – – – 9 – 30 49 50–51 43 47 42 44 50 11 12 9 11 – 10 35 50 52 44 48 – 45 60 12 13 10–11 12 10 11 40 – – 45 49 43 46 70 – – 12 13 11 12 45 51 53 46–47 50–51 – 47 80 13 14 13 14 12 13 50 52 54 48 – 44 48 90 14 – 14 – 14 55 53 55 49 52 45–46 49 95 15 15 15 15 ≥ 13 15 60 54 – 50 53–55 47–48 50 <16 years of education, less than 16 years of education; ≥16 years of education, 65 55–56 56 51–52 – 49 51–52 16 years of education or more. 70 57 57–58 53 56 50 53–54 75 – 59–61 54 57–59 51–53 55 80 58 62 55 60–61 54 56–57 The interaction Delay × Group also resulted significant 85 59–60 63 56–59 62 55–56 58 (F = 6.99, p = 0.001, ηp2 = 0.05), indicating differences in the 90 61–64 64–65 60–64 63 57–61 59–60 RAVLT performance between the three groups depending on 95 65 66 65 64–65 62 61 the deferred lapse after initial learning. As Figure 1 illustrates, 98 – ≥67 – ≥66 – ≥62 the decrease in the number of evoked words between the <16 years of education, less than 16 years of education; ≥16 years of education, 25-min and 1-week delayed recall measures depends on the 16 years of education or more. application modality. Post hoc analysis revealed no differences between the three TABLE 8 | Normative data for women performance on RAVLT Delayed Recall. groups in the 25-min Delayed Recall (F = 0.44, p = 0.64), as we previously demonstrated through the one-way ANOVA, but 1- Women week delayed measures showed significant differences (F = 5.09, p = 0.01). Specifically, the participants of Group III, who were 41–48 years 49–57 years 58–65 years explicitly warned about the 1-week recall, performed significantly Percentile <16 yrs ≥16 yrs <16 yrs ≥16 yrs <16 yrs ≥16 yrs better than the participants of Group I (p = 0.04) and Group II educ educ educ educ educ educ (p = 0.01) (see Figure 1). Finally, multiple regression linear analyses were carried out 2 5 6 – 6 – 6 for each modality group to determine the contribution of 5 6 7–8 8 8 5–6 7 demographic variables as predictors of 1-week delayed measures. 10 7–8 9–10 9 9 7–8 8 Concerning Group I, the regression model [F(3,79) = 6.97; 20 9 11 10 10 9 9 p < 0.001] reflected that age (β = −0.42; p < 0.001) and gender 30 10 12 – 11 10 10 (β = −0.20; p = 0.04) explained 21% of the variance of 1-week 40 – – 11 – 11 11 Delayed Recall. Men and older participants tended to recall fewer 50 11 13 – 12 – – words after 1-week. For their Recognition score [F(3,79) = 3.14; 60 – – 12 13 12 – p = 0.02; R2 = 0.12], only age appeared as a significant predictor 70 12 14 13 – 13 12 (β = −0.27; p = 0.02). Also, 12% of the variance of Forgetting Rate 80 13 – 14 14 14 13 [F(3,79) = 3.51; p = 0.20] was predicted by age (β = 0.31; p = 0.01), 90 14 – – – – 14 showing that younger participants tend to forget less. 95 15 15 15 15 15 15 Regarding Group II, the regression model for 1-week Delayed <16 years of education, less than 16 years of education; ≥16 years of education, Recall [F(3,81) = 6.34; p < 0.001; R2 = 0.19] indicated that gender 16 years of education or more. was the only predictor (β = −0.44; p < 0.001), and Recognition was explained [F(3,81) = 2.96; p = 0.04; R2 = 0.10] by years of recall trial 25 min after initial learning (Delayed Recall), resulting education (β = 0.28; p = 0.01; R2 = 0.10). These findings reflect statistically significant in the calculated mixed ANOVA [Delay that women within this group tend to spontaneously recall more main effect: F(1121.91), p < 0.001, ηp2 = 0.82]. words, while those with lower educational levels obtain poorer Frontiers in Aging Neuroscience | www.frontiersin.org 8 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting TABLE 10 | Normative data for Recognition, Errors, Learning Over Trial (LOT), and Forgetting Rate on RAVLT. Percentiles Recognition Repeated Intrusions Repeated Intrusions LOT Forgetting Rate 2 ≤11 ≥14 ≥3 ≥4 7–9 ≥45 5 12 11–13 2 2–3 10–12 44–37 10 13 10 – 1 13–15 36–31 20 – 8–9 – – 16 21–30 30 14 6–7 – 0 17–18 17–20 40 – 5 1 19–20 13–16 50 15 4 0 21 9–12 60 – 22–23 7–8 70 3 24–25 – 80 2 26–28 0 90 1 29–31 −1 to −7 95 0 32–34 −8 to −10 98 ≥35 ≤-11 LOT, Learning Over Trial or Immediate Total Recall - (5 × Trial I). performance in the recognition task. With respect to Forgetting population, which allow comparisons between risk population Rate [F(3,81) = 3.90; p = 0.01; R2 = 0.01], gender was the only (i.e., APOE carriers) and their corresponding normative values. statistically significant predictor (β = 0.35; p < 0.001), confirming Thus, a better understanding of preclinical and prodromal stages that men tend to evoke fewer words after 1-week. of AD would be obtained so early therapeutic interventions could Finally, for Group III, the 1-week Delayed Recall regression be applied to prevent disability (Ritchie et al., 2017). model [F(3,79) = 3.03; p = 0.03; R2 = 0.10] was the only statistically significant, with gender being the predictor of the scores (β = −0.22; p = 0.04). The findings reflected that men TABLE 11 | Demographic characteristics of the long-term forgetting (LTF) sample. tended to perform worse. Administration modality DISCUSSION Group I Group II Group III n = 83 n = 85 n = 83 Episodic memory (EM) assessment is a fundamental pillar in the study of brain aging that provides premature signs n % n % n % of neurodegenerative disorders due to its multidimensional Gender Man 40 48.19 40 47.06 40 48.19 complexity and high vulnerability to disease (Pause et al., 2013; Woman 43 51.81 45 52.94 43 51.81 Polcher et al., 2017). Even in cognitively normal older adults, Age Range 41–49 33 36.14 26 30.59 30 34.94 an association between amyloid burden and episodic memory 50–57 26 34.94 40 47.06 27 33.73 performance has been detected. Therefore, the earliest preclinical 58–65 24 28.92 19 22.35 26 31.33 stages of dementia may have subtle but measurable effects on Educational Level 0–15 years 26 31.33 24 28.24 26 31.33 cognition that could help identify diseases prematurely (Hedden ≥16 years 57 68.67 61 71.76 57 68.67 et al., 2012; Duke Han et al., 2017; Baker et al., 2018). In this sense, the RAVLT is a potential sensitive tool to measure changes associated with abnormal aging (Estévez-González et al., TABLE 12 | Performance on LTF assessment. 2003; Drolet et al., 2014; Marchand et al., 2017; Moradi et al., Group Min Max Mean SD 2017). Thus, this test has been widely used in longitudinal studies to help describe the memory trajectory in cognitively normal and 1-week Delayed Recall I 1 14 6.48 3.01 MCI subjects as an indicator of dementia risk. Sum Trials I to II 2 15 6.24 2.80 V, the RAVLT learning score (V-I), and 25-min delayed recall III 1 15 7.64 3.28 play an important role in the prediction of progression from MCI 1-week Delayed Recognition I 6 15 12.55 1.97 to AD dementia (Crane et al., 2012; Wang et al., 2016; Li et al., II 7 15 12.58 1.85 2017; Li et al., 2018; Ding et al., 2019; Abraham et al., 2020). Also, III 6 15 12.77 1.91 error scores have been associated with different types and stages Intrusions I 0 5 0.66 1.11 of cognitive impairment (Baños et al., 2005; Thomas et al., 2018; II 0 3 0.67 0.88 Weitzner et al., 2020). For this reason, considering that preclinical III 0 5 0.69 0.96 changes appear up to two decades before the diagnosis of AD 1-week Forgetting Rate I 0 87.5 46 20.22 dementia (Chipi et al., 2019), it is relevant and indispensable II 0 83.33 46.29 17.50 to have sensitive tests with normative data for the middle-aged III −9.09 88.89 35.80 20.67 Frontiers in Aging Neuroscience | www.frontiersin.org 9 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting attention, another relative to verbal learning, and the last one with inaccurate recall loaded (only intrusions), consistent with our findings. Finally, Weitzner et al. (2020) found a factorial solution with a good fit in a middle-aged sample similar to the one we described. However, they, in addition to the error measures, included other process scores. They defined the Attention/Learning factor that partially corresponds to our first component (the Attention and Memory span) even though we did not consider serial position scores. The Memory factor, related to short-term memory, long-term memory, and recognition, is comparable to our second component (Memory and Learning). Inaccurate Memory corresponds to our Repeated and Intrusion factors. Our findings highlight the value of using multifactorial FIGURE 1 | Performance on 25-min Delayed Recall and 1-week Delayed measures, including execution errors, to facilitate the Recall between modality groups. Error bars represent SEM. characterization of healthy and clinical populations through an empirical framework for diagnosis, rehabilitation, or research purposes (Vakil and Blachstein, 1993; Baños et al., 2005; Weitzner et al., 2020). The quantification of error scores is an Rey Auditory Verbal Learning Test added worth, considering previous evidence of its predictive Validation value of abnormal aging (Bezdicek et al., 2014; Ferreira Correia Research on RAVLT construct validity through factor structure and Campagna Osorio, 2014; Libon et al., 2015; Thomas et al., has been scarce, and most of the studies lacked the inclusion of 2018; Weitzner et al., 2020). Thus, we firmly recommend error scores. It has been recommended to replicate the structural including these measures and other process measures in clinical analysis in different samples and include repeated words and and longitudinal studies to confirm their discriminative and intrusions, considering their value (Vakil and Blachstein, 1993; predictive capacity in early aging. Baños et al., 2005; Weitzner et al., 2020). Considering that no Regarding convergent validity, Fichman et al. (2010) found a test validation has been performed in the healthy Spanish middle- positive linear relation between the RAVLT and a memory task aged population and the addition of error scores, it was necessary of the Brief Cognitive Screening Battery (BCSB), even though the to perform an EFA to find out how many factors can be extracted input of this test is visual. In the present study, we found a positive after including error scores. Then, to finally determine the factor correlation with the S-FNAME Exam, an associative memory structure of the test and its goodness of fit, we carried out CFA test of verbal-visual stimuli that require immediate and delayed through the rival model strategy (Hair et al., 1999; Lloret-Segura recall, with more demanding encoding, storage, and retrieval et al., 2014). strategies than BCSB used in previous research. As a limitation, Our CFA results showed that the model with the best goodness The FNAME and the RAVLT are instruments that correspond to of fit is the four-factor model, which includes the first component different paradigms; both cover verbal stimuli, but the FNAME associated with the Attention and Memory span (Trials I and II), is a visual EM test. Therefore, we recommend assessing the the second one related to Memory and Learning (Trials III, IV, convergent validity using verbal auditory EM tests in Spanish for and V, Delayed Recall, and Recognition), and the third and fourth future research. referring to error measures such as Perseverations and Intrusions, According to divergent validity, weak positive correlations respectively. The resulting factor structure corresponds to the between the RAVLT and non-amnesic tests have been theoretical framework and previous findings, although some of demonstrated. These results were also significantly lower those studies did not include error measures, unlike our proposal. than the linear relation with the S-FNAME Exam. Our findings Vakil and Blachstein (1993) found a factor structure formed are similar to previous results (de Paula et al., 2012), highlighting by the three main memory processes (Squire, 1982; Lezak that the RAVLT assesses a different construct than those et al., 2012): acquisition, storage, and retrieval. Acquisition measured by widely used non-amnesic tests. (Trial I) is related to the attention and memory span, Retrieval includes the learning score and delayed recall trial, and Storage relates to the recognition score only. This solution resembles the one shown in the present study, although we found that Rey Auditory Verbal Learning Test Norms the recognition score loaded within the Memory and Learning Development factor. Vakil and Blachstein (1993) emphasized that memory In line with previous findings, women outperformed males on components, in normal subjects, would be strongly related to RAVLT scores, especially on immediate total recall and delayed each other, explaining why the recognition and consolidation recall (Gale et al., 2007; Messinis et al., 2007; Speer et al., 2014). processes grouped. Delayed RAVLT recall in elderly samples suggests that men In addition, Baños et al. (2005) obtained a three-factor remain stable across age ranges, while women show a subtle solution that includes a significant factor indexing auditory decline. Differences in verbal memory by sex may be related to Frontiers in Aging Neuroscience | www.frontiersin.org 10 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting distinct encoding and consolidation strategies (Gale et al., 2007; risk and education, that strongest prediction of low risk of Zhang et al., 2017). dementia is obtained by the stratification in low (non-tertiary It has been widely demonstrated that age has an inverse education) and high (tertiary education) educational levels. relation with EM (Schoenberg et al., 2006; Fichman et al., 2010; Different RAVLT versions with variations administration Vakil et al., 2010; Marqués et al., 2013; Ferreira Correia and procedures have been developed (Schmidt, 1996; Estévez- Campagna Osorio, 2014; Speer et al., 2014; Lavoie et al., 2018). González et al., 2003; Lezak et al., 2012; Bezdicek et al., 2014; However, not all components of EM change homogeneously with Cavaco et al., 2015; Puerta Lopera et al., 2018). It is crucial to age. Older adults reflect recall impairments but typically showed notice that the Spanish RAVLT version used lacks Interference minimal age decline in recognition tasks (Healey and Kahana, List B, which provides information on the effect of interference 2016). This divergence in trajectories is consistent with our on memory consolidation. This version is frequently used in RAVLT findings: Age does not significantly influence recognition, Spain (Perea Bartolomé et al., 2000; Estévez-González et al., 2003; but affects immediate and delayed recall. Regarding errors, unlike García-Rudolph et al., 2020; Albu et al., 2021); however, this may our results, Baños et al. (2005) reported lower inaccurate recall be considered a limitation and should be taken into account when scores for younger individuals. using these normative data. As Estévez-González et al. (2003) Finally, the influence of education on RAVLT scores is widely and Messinis et al. (2007) mentioned, the RAVLT application described. The higher educational level has been associated with methods have not been uniform in the literature, sometimes better performance as we found in the present study, although restricting the use of normative data. some previous studies did not find the significative contribution of this variable (Messinis et al., 2007; Fichman et al., 2010; Long-Term Forgetting Assessment Ferreira Correia and Campagna Osorio, 2014; Lavoie et al., 2018). Beyond these limitations discussed above, a relatively novel and Understanding the effect of demographic variables on RAVLT potentially vital aspect of this study is the introduction of LTF and verbal episodic memory is essential for designing prevention, assessment, a sensitive marker of memory decline in abnormal stimulation, and rehabilitation protocols in aging. aging (Reiman, 2018; Weston et al., 2018; Zimmermann Neuropsychological assessment using standardized and Butler, 2018; Butler et al., 2019; Wearn et al., 2020). neurocognitive measures is a priority within clinical and Clinical and theoretical interest in LTF has emerged recently, research practice. Thus, developing sensitive measures to and it is recommended to use more demanding cognitive identify the early stages of neurocognitive disorders is a current instruments, including LTF measures, to detect the earliest need to plan a comprehensive treatment (Moradi et al., 2017). cognitive manifestations in presymptomatic AD (Butler et al., There is still a need to develop and publish normative data for 2019; Tort-Merino et al., 2021a). Spanish-speaking subjects, considering that cultural, linguistic, Considering methodological issues exposed by Elliott et al. or sociodemographic variables could affect a cognitive profile (2014) associated with the procedures and rehearsal effects and vary between different populations (Peña-Casanova et al., in longitudinal follow-ups, the present study compared the 2012; Del Pino et al., 2015). performance on RALTV, adding 1-week delayed retrieval after The exclusion criteria of the present study were chosen to the initial learning, under three different procedures. Within the ensure a sample of cognitively healthy subjects. However, the framework of long-term forgetting assessment, this is the first limitation of the study is that it lacks subjective complaints and study, to our knowledge, that shows evidence about the difference biological measures that could identify participants within the between warning the participants or not about the delayed recall preclinical phase of AD. Regarding the age range, it is essential with a more extended period since the initial learning. to highlight that the objective of the present study was to validate Results revealed that, while there were no differences in the the Spanish version of the RAVLT and generate normative immediate and 25-min delayed recall between the three groups, data for the middle-aged population, considering the lack of the participants informed about the 1-week recall test showed psychometric studies for this population in Spain. Marqués et al. less LTF, extending prior findings that revealed the influence of (2013) published normative data for a Spanish version of the test expectations have on the establishment of new memories RAVLT in older people, but it is crucial to have a memory test in long-term storage (Shimizu, 1996; Szpunar et al., 2007). that allows early measurement of changes associated with aging. The “expectancy manipulation” was done after initial learning, Also, we strongly recommend replicating the factorial structure excluding the possible effect on encoding. These results suggest using an older aged sample and extending normative data and that consolidation is associated with top-down modulation evidence of validity in other age ranges. by knowing the value and utility of the previously learned Our sample size was large enough to be representative information (Wamsley et al., 2016). Retroactive interference and met the criteria for validation studies and factor analysis related to mental effort and learning of new material may explain requirements (Hair et al., 1999; Speer et al., 2014). Nevertheless, the decay of memory due to the participation of the hippocampus the overrepresentation of highly educated subjects in our in the consolidation phase (Hoefeijzers et al., 2013; Brawn cognitively normal sample should be considered a limitation. et al., 2018). Retroactive interference was not controlled in the This unbalanced distribution by the educational level is frequent present study and should be considered in future LTF research. in other Spanish normative studies (Del Pino et al., 2015) and Also, it is essential to consider that, although the participants has been reported in previous RAVLT validation projects (Lavoie were asked not to practice, there is a possibility that they have et al., 2018; Weitzner et al., 2020). Then et al. (2016) suggested, done so. Information retrieval permits memory integration with based on the exploration of the relationship between dementia stored neocortical knowledge, making memory less hippocampus Frontiers in Aging Neuroscience | www.frontiersin.org 11 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting dependent and more reachable for recall in the future (Antony to develop predictive models of healthy and pathological aging. et al., 2017). That is why it is challenging to design a genuinely It is recommended to conduct studies with subjects within the unexpected test more than one time and to control the effect preclinical phase of AD, and patients with MCI and AD diagnosis, of rehearsal through a longitudinal assessment (Lee et al., 2017; to detect its sensitivity, specificity, and precise cut-off points Wyble and Chen, 2017). that reflect a cognitive decline (Reiman, 2018). Also, we suggest On the other hand, the performance of Group I subjects, including APOE status or other AD markers in plasma or CSF to who were unaware of the LTF assessment, could be a reliable characterize the sample and determine the relationship between measure of incidental long-term recall, considering that all of RAVLT and these biological measures. them claimed not to have practiced, as previously mentioned. It would be interesting to determine the performance of these subjects on repeated longitudinal measures to expand knowledge DATA AVAILABILITY STATEMENT regarding the LTF construct, since given the delayed recall experience, possible test expectations would be generated. The raw data supporting the conclusions of this article will be In any of the three modalities, we present a measurement made available by the authors, without undue reservation. approach that encompasses both delayed free recall and a recognition task. These memory paradigms are fundamental to understanding LTF because the cognitive profiles of clinical ETHICS STATEMENT samples may reflect differences in performance on these two tasks, as noted by Elliott et al. (2014). The studies involving human participants were reviewed and It is relevant to point out that previous LTF research (Weston approved by Comité d’Ètica i Investigació Clínica de la Unió et al., 2018; Zimmermann and Butler, 2018) has fixed a minimum Catalana d’Hospitals. The patients/participants provided their acquired criterion (accuracy of 80% over a maximum of 10 written informed consent to participate in this study. initial trials), considering forgetting rates are inherently related to learning performance. Considering that this administration procedure would have altered the RAVLT version used in AUTHOR CONTRIBUTIONS this validation study, we did not follow this criterion, as described in the Section “Materials and Methods.” However, we GC, DB-F, and VA-S participated in drafting the manuscript and excluded subjects with objective cognitive deficits during the made substantial contributions to the analysis and interpretation neuropsychological assessment, including the Immediate Total of data. CP-G, GC, and VA-S made substantial contribution to the Recall. Furthermore, we demonstrated no differences in the acquisition of data. All authors made substantial contributions Immediate Total Recall score between the three groups, and their to conception, design, and interpretation of data, contributed performance reflected an accuracy of 70%. to revising it critically for important intellectual content, and This study highlights the importance of knowing the effect of approved the submitted version. different methodologies in memory assessment and the urgency of having agreed on methodological procedures to interpret LTF data and compare findings without bias. More studies FUNDING on LTF should be done in the preclinical stages of abnormal aging pathologies to confirm their predictive value and describe This research leading to these results have received possible relations with other biomarkers (Wearn et al., 2020). funding from “la Caixa” Foundation (Grant agreement n◦ Hence, it is crucial to design or adapt valid LTF instruments LCF/PR/PR16/11110004) and also from Institut Guttmann and (Wearn et al., 2020), even for the middle-aged population, Fundació Abertis. DB-F was funded by the Spanish Ministry of establishing best practices that minimize potential rehearsal and Science, Innovation and Universities (RTI2018-095181-B-C21) learning effects associated with longitudinal assessments. Also, and an Institut Català de Recercai Estudis Avançats ICREA it is urgent to provide normative data with a broader sample, Acadèmia 2019 award. A. Pascual-Leone is a co-founder of Linus including long-term recall measures after 4 weeks or 3 months, Health and TI Solutions AG; serves on the scientific advisory considering previous findings that relate these measures to AD boards for Starlab Neuroscience, Neuroelectrics, Magstim Inc., biomarkers (Tort-Merino et al., 2017, 2021a,b; Wearn et al., and MedRhythms; and is listed as an inventor on several issued 2020). The availability of LTF instruments would be beneficial in and pending patents on the real-time integration of non-invasive improving the sensitivity of conventional EM tests in both aging brain stimulation with electroencephalography and magnetic research and clinical practices. resonance imaging. JT was partly supported by Fundació Joan Ribas Araquistain_Fjra, AGAUR, Agència de Gestió d’Ajuts Universitaris i de Recerca (2018 PROD 00172), FUNDACIó LA CONCLUSION MARATó DE TV3 (201735.10), and the European Commission (Call H2020-SC1-2016-2017_RIA_777107), Ad-Salutem In conclusion, the validation and normalization of the RAVLT in Institute, Sodexo SA, ICA Informática y Comunicaciones a Spanish sample and the proposal of an LTF measure using this Avanzadas, Neuroelectrics, Corporació Catalana de Mitjans widely known instrument are extremely valuable, considering Audiovisuals, Club Metropolitan, Casa Ametller, and Agència that the RAVLT is a helpful tool, along with other biomarkers, de Qualitati Avaluació Sanitàries de Catalunya-AQuAS that Frontiers in Aging Neuroscience | www.frontiersin.org 12 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting are commercial companies supporting the Barcelona Brain Avanzadas, Neuroelectrics, Corporació Catalana de Mitjans Health Initiative and this study. The funders were not involved in Audiovisuals, Club Metropolitan, Casa Ametller, and Agència the study design, collection, analysis, interpretation of data, the de Qualitati Avaluació Sanitàries de Catalunya-AQuAS) for their writing of this article or the decision to submit it for publication. vital collaboration. ACKNOWLEDGMENTS SUPPLEMENTARY MATERIAL Special thanks are extended to all BBHI participants, the The Supplementary Material for this article can be found project team and research assistants, and other partners (Ad- online at: https://www.frontiersin.org/articles/10.3389/fnagi. Salutem Institute, Sodexo, I.C.A. Informàtica y Comunicaciones 2022.809019/full#supplementary-material REFERENCES Butler, C., Gilboa, A., and Miller, L. (2019). Accelerated long-term forgetting. Cortex 110, 1–4. doi: 10.1016/j.cortex.2018.12.009 Abraham, M., Seidenberg, M., Kelly, D. A., Nielson, K. A., Woodard, Cattaneo, G., Bartrés-Faz, D., Morris, T. P., Sánchez, J. S., Macià, D., Tarrero, C., J. L., and Carson Smith, J. (2020). Episodic Memory and Hippocampal et al. (2018). The Barcelona Brain Health Initiative: A Cohort Study to Define Volume Predict 5-Year Mild Cognitive Impairment Conversion in Healthy and Promote Determinants of Brain Health. Front. Aging Neurosci. 10:321. Apolipoprotein ε4 Carriers. J. Int. Neuropsychol. Soc. 26, 733–738. doi: 10.1017/ doi: 10.3389/fnagi.2018.00321 S1355617720000181 Cavaco, S., Gonçalves, A., Pinto, C., Almeida, E., Gomes, F., Moreira, I., Albu, S., Zozaya, N. R., Murillo, N., García-Molina, A., Chacón, C. A. F., et al. (2015). Auditory verbal learning test in a large nonclinical Portuguese and Kumru, H. (2021). What’s going on following acute covid-19? population. Appl. Neuropsychol. 22, 321–331. doi: 10.1080/23279095.2014. Clinical characteristics of patients in an out-patient rehabilitation program. 927767 NeuroRehabilitation 48, 469–480. doi: 10.3233/nre-210025 Chipi, E., Salvadori, N., Farotti, L., and Parnetti, L. (2019). Biomarker-based Alegret, M., Valero, S., Ortega, G., Espinosa, A., Sanabria, A., Hernández, I., signature of alzheimer’s disease in pre-MCI individuals. Brain Sci. 9, 1–22. et al. (2015). Validation of the Spanish version of the face name associative doi: 10.3390/brainsci9090213 memory exam (S-FNAME) in cognitively normal older individuals. Arch. Clin. Crane, P. K., Carle, A., Gibbons, L. E., Insel, P., Mackin, R. S., Gross, A., et al. Neuropsychol. 30, 712–720. doi: 10.1093/arclin/acv050 (2012). Development and assessment of a composite score for memory in the Alviarez-Schulze, V., Cattaneo, G., Pachón-García, C., Tormos-Muñoz, J. M., Alzheimer’s Disease Neuroimaging Initiative (ADNI). Brain Imaging Behav. 6, Alegret, M., Pascual-Leone, A., et al. (2022). Validation and normative data of 502–516. doi: 10.1007/s11682-012-9186-z the Spanish version of the Face-Name Associative Memory Exam (S-FNAME). Davidson, M., Dorris, L., O’Regan, M., and Zuberi, S. M. (2007). Memory J. Int. Neuropsychol. Soc.: JINS 28, 74–84. doi: 10.1017/S1355617721000084 consolidation and accelerated forgetting in children with idiopathic generalized Antony, J. W., Ferreira, C. S., Norman, K. A., and Wimber, M. (2017). Retrieval as epilepsy. Epilepsy Behav. E B 11, 394–400. doi: 10.1016/j.yebeh.2007.05.004 a fast route for consolidation. Trends Cogn. Sci. 21, 573–576. doi: 10.1016/j.tics. de Paula, J. J., Melo, L. P. C., Nicolato, R., de Moraes, E. N., Bicalho, M. A., 2017.05.001.Retrieval Hamdan, A. C., et al. (2012). Reliability and construct validity of the Rey- Atherton, K. E., Filippini, N., Zeman, A. Z. J., Nobre, A. C., and Butler, C. R. (2019). Auditory verbal learning test in Brazilian elders. Arch. Clin. Psychiatry 39, Encoding-related brain activity and accelerated forgetting in transient epileptic 19–23. doi: 10.1590/S0101-60832012000100004 amnesia. Cortex 110, 127–140. doi: 10.1016/j.cortex.2018.04.015 Del Pino, R., Peña, J., Schretlen, D., Ibarretxe-Bilbao, N., and Ojeda, N. (2015). Baddeley, A., Atkinson, A., Kemp, S., and Allen, R. (2019). The problem of Multisite study for norming and standardizing neuropsychological instruments detecting long-term forgetting: Evidence from the Crimes Test and the Four in healthy people for Spanish population: methods and characteristics of Doors Test. Cortex 110, 69–79. doi: 10.1016/j.cortex.2018.01.017 Normacog project. Rev. Neurol. 61, 57–65. Baker, J. E., Lim, Y. Y., Jaeger, J., Ames, D., Lautenschlager, N. T., Robertson, J., et al. Ding, X., Charnigo, R. J., Schmitt, F. A., Kryscio, R. J., Abner, E. L., Alzheimer’s (2018). Episodic memory and learning dysfunction over an 18-month period in Disease, et al. (2019). Evaluating trajectories of episodic memory in normal preclinical and prodromal Alzheimer’s disease. J. Alzheimer’s Dis. 65, 977–988. cognition and mild cognitive impairment: Results from ADNI. PLoS One doi: 10.3233/JAD-180344 14:e0212435. doi: 10.1371/journal.pone.0212435 Baños, J. H., Elliott, T. R., and Schmitt, M. (2005). Factor Structure of the Rey Drolet, V., Vallet, G. T., Imbeault, H., Lecomte, S., Limoges, F., Joubert, S., et al. Auditory Verbal Learning Test in Adults With Spinal Cord Injury. Rehabil. (2014). Comparaison des performances à l’épreuve des 15 mots de Rey et Psychol. 50, 375–380. doi: 10.1037/0090-5550.50.4.375 au RL/RI 16 dans le vieillissement normal et la démence de type Alzheimer. Batista, J. M., and Coenders, G. (2000). Modelos de ecuacionesestructurales (modelos Geriatr. Psychol. Neuropsychiatr. Vieil. 12, 218–226. doi: 10.1684/pnv.2014. para el análisis de relacionescausales). Madrid: Hespérides. 0469 Bezdicek, O., Stepankova, H., Moták, L., Axelrod, B. N., Woodard, J. L., Preiss, M., Duke Han, S., Nguyen, C. P., Stricker, N. H., and Nation, D. A. (2017). Detectable et al. (2014). Czech version of Rey Auditory Verbal Learning test: normative Neuropsychological Differences in Early Preclinical Alzheimer’s Disease: A data. Neuropsychology, development, and cognition. Sect. B Aging Neuropsychol. Meta-Analysis. Neuropsychol. Rev. 27, 305–325. doi: 10.1007/s11065-017- Cognit. 21, 693–721. doi: 10.1080/13825585.2013.865699 9345-5 Boraxbekk, C. J., Lundquist, A., Nordin, A., Nyberg, L., Nilsson, L. G., and Elliott, G., Isaac, C. L., and Muhlert, N. (2014). Measuring forgetting: A critical Adolfsson, R. (2015). Free Recall Episodic Memory Performance Predicts review of accelerated long-term forgetting studies. Cortex 54, 16–32. doi: 10. Dementia Ten Years prior to Clinical Diagnosis: Findings from the Betula 1016/j.cortex.2014.02.001 Longitudinal Study. Dement. Geriatr. Cogn. Dis. Extra 5, 191–202. doi: 10.1159/ Estévez-González, A., Kulisevsky, J., Boltes, A., Otermín, P., and García-Sánchez, 000381535 C. (2003). Rey verbal learning test is a useful tool for differential diagnosis in Brawn, T. P., Nusbaum, H. C., and Margoliash, D. (2018). Differential development the preclinical phase of Alzheimer’s disease: Comparison with mild cognitive of retroactive and proactive interference during post-learning wakefulness. impairment and normal aging. Int. J. Geriatr. Psychiatry 18, 1021–1028. doi: Learn. Mem. 25, 325–329. doi: 10.1101/lm.046573.117 10.1002/gps.1010 Butler, C. R., Graham, K. S., Hodges, J. R., Kapur, N., Wardlaw, J. M., and Zeman, Ferreira Correia, A., and Campagna Osorio, I. (2014). The rey auditory verbal A. Z. J. (2007). The syndrome of transient epileptic amnesia. Ann. Neurol. learning test: Normative data developed for the venezuelan population. Arch. 61:587e598. Clin. Neuropsychol. 29, 206–215. doi: 10.1093/arclin/act070 Frontiers in Aging Neuroscience | www.frontiersin.org 13 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting Fichman, H. C., Dias, L. B. T., Fernandes, C. S., Lourenço, R., Caramelli, P., and prospective study of REM sleep behavior disorder. Sleep 40:zsw014. doi: 10. Nitrini, R. (2010). Normative data and construct validity of the rey auditory 1093/sleep/zsw014 verbal learning test in a Brazilian elderly population. Psychol. Neurosci. 3, 79–84. Marqués, N. O., Caro, I. A., Uterga Valiente, J. M., and Rodríguez, S. M. (2013). doi: 10.3922/j.psns.2010.1.010 Normative data for a Spanish version of the Rey Auditory-Verbal Learning Test Gale, S. D., Baxter, L., Connor, D. J., Herring, A., and Comer, J. (2007). Sex in older people. Span. J. Psychol. 16, 1–12. doi: 10.1017/sjp.2013.63 differences on the Rey Auditory Verbal Learning Test and the Brief Visuospatial Mayes, A. R., Hunkin, N. M., Isaac, C., and Muhlert, N. (2019). Are there distinct Memory Test-Revised in the elderly: Normative data in 172 participants. J. Clin. forms of accelerated forgetting and, if so, why? Cortex 110, 115–126. doi: 10. Exp. Neuropsychol. 29, 561–567. doi: 10.1080/13803390600864760 1016/j.cortex.2018.04.005 García-Rudolph, A., García-Molina, A., Opisso, E., Tormos, J. M., and Bernabeu, Messinis, L., Tsakona, I., Malefaki, S., and Papathanasopoulos, P. (2007). M. (2020). Cognition assessments to predict inpatient falls in a subacute stroke Normative data and discriminant validity of Rey’s Verbal Learning Test for the rehabilitation setting. Topics Stroke Rehabil. 2020, 1–9. doi: 10.1080/10749357. Greek adult population. Arch. Clin. Neuropsychol. 22, 739–752. doi: 10.1016/j. 2020.1765660 acn.2007.06.002 Geurts, S., van der Werf, S. P., and Kessels, R. P. C. (2015). Accelerated forgetting? Montero, L. D., and Cáceres, M. (2017). Indicadores cognitivos en pruebas de An evaluation on the use of long-term forgetting rates in patients with memory memoria verbal: prevalencia en población con diferentes perfiles de deterioro problems. Front. Psychol. 6:1–9. doi: 10.3389/fpsyg.2015.00752 cognitivo. Rev. Neuropsicol. Latinoam. 9, 56–64. doi: 10.5579/rnl.2017.0317 Hair, J., Anderson, R., Tatham, R., and Black, W. (1999). AnálisisMultivariante, 5th. Moradi, E., Hallikainen, I., Hänninen, T., and Tohka, J. (2017). Rey’s Auditory Edn. Madrid: Prentice Hall. Verbal Learning Test scores can be predicted from whole brain MRI in Healey, M., and Kahana, M. J. (2016). A four-component model of age-related Alzheimer’s disease. NeuroImage Clin. 13, 415–427. doi: 10.1016/j.nicl.2016.12. memory change. Psychol. Rev. 123, 23–69. doi: 10.1037/rev0000015 011 Hedden, T., Mormino, E. C., Amariglio, R. E., Younger, A. P., Schultz, A. P., Muhlert, N., Milton, F., Butler, C. R., Kapur, N., and Zeman, A. Z. (2010). Becker, J. A., et al. (2012). Cognitive profile of amyloid burden and white Accelerated forgetting of real-life events in Transient Epileptic Amnesia. matter hyperintensities in cognitively normal older adults. J. Neurosci. 32, Neuropsychologia 48, 3235–3244. doi: 10.1016/j.neuropsychologia.2010.07.001 16233–16242. doi: 10.1523/JNEUROSCI.2462-12.2012 Nyberg, L., and Pudas, S. (2019). Successful Memory Aging. Annu. Rev. Psychol. 70, Hoefeijzers, S., Dewar, M., Della Sala, S., Zeman, A., and Butler, C. (2013). 219–243. doi: 10.1146/annurev-psych-010418-103052 Accelerated long-term forgetting in transient epileptic amnesia: An acquisition Pause, B. M., Zlomuzica, A., Kinugawa, K., Mariani, J., Pietrowsky, R., and Dere, or consolidation deficit? Neuropsychologia 51, 1549–1555. doi: 10.1016/j. E. (2013). Perspectives on episodic-like and episodic memory. Front. Behav. neuropsychologia.2013.04.017 Neurosci. 7:1–12. doi: 10.3389/fnbeh.2013.00033 Kormas, C., Megalokonomou, A., Zalonis, I., Evdokimidis, I., Kapaki, E., and Peña-Casanova, J., Casals-Coll, M., Quintana, M., Sánchez-Benavides, G., Potagas, C. (2018). Development of the Greek version of the Face Name Rognoni, T., Calvo, L., et al. (2012). Estudiosnormativosespañolesen población Associative Memory Exam (GR-FNAME12) in cognitively normal elderly adultajoven (Proyecto NEURONORMA jóvenes): métodos y características individuals. Clin. Neuropsychol. 32, 152–163. doi: 10.1080/13854046.2018. de la muestra [Spanish normative studies in a young adult population 1495270 (NEURONORMA young adults Project): methods and characteristics of the Lavoie, M., Bherer, L., Joubert, S., Gagnon, J. F., Blanchet, S., Rouleau, I., et al. sample]. Neurología 27, 253–260. doi: 10.1016/j.nrl.2011.12.019 (2018). Normative data for the Rey Auditory Verbal Learning Test in the Perea Bartolomé, M. V., Ladera Fernández, V., and Morales Ramos, F. (2000). older French-Quebec population. Clin. Neuropsychol. 32, 15–28. doi: 10.1080/ Aprendizaje verbal en el traumatismo craneoencefálico. Psicothema 12, 353– 13854046.2018.1429670 359. Lee, J., Nader, K., and Schiller, D. (2017). An Update on Memory Reconsolidation Polcher, A., Frommann, I., Koppara, A., Wolfsgruber, S., Jessen, F., and Wagner, Updating. Trends Cognit. Sci. 21, 531–545. doi: 10.1016/j.tics.2017.04.006 M. (2017). Face-Name Associative Recognition Deficits in Subjective Cognitive Lezak, M., Howieson, D., Bigler, E., and Tranel, D. (2012). Decline and Mild Cognitive Impairment. J. Alzheimer’s Dis. 56, 1185–1196. NeuropsychologicalAssessment, 5th Edn. New York, NY: Oxford University doi: 10.3233/JAD-160637 Press. Puerta Lopera, I. C., Dussan Lubert, C., Montoya Londoño, D. M., and Landinez Li, K., Chan, W., Doody, R. S., Quinn, J., Luo, S., Alzheimer’s Disease, et al. (2017). Martinez, D. A. (2018). Datos normativos y estandarización de un protocolo Prediction of Conversion to Alzheimer’s Disease with Longitudinal Measures de pruebas neuropsicológicas para la evaluación de la memoria en estudiantes and Time-To-Event Data. J. Alzheimer’s Dis. 58, 361–371. doi: 10.3233/JAD- universitarios. Psychologia 12, 23–35. doi: 10.21500/19002386.3260 161201 Reiman, E. M. (2018). Long-term forgetting in preclinical Alzheimer’s disease. Li, K., O’Brien, R., Lutz, M., Luo, S., Alzheimer’s Disease and Neuroimaging Lancet Neurol. 17:104. doi: 10.1016/S1474-4422(17)30458-1 Initiative (2018). A prognostic model of Alzheimer’s disease relying on multiple Rey, A. (1964). L’examen Clinique enPsychologie, 2th Edn. Paris: Presses longitudinal measures and time-to-event data. Alzheimer’s Dement. 14, 644– Universitaires de France. 651. doi: 10.1016/j.jalz.2017.11.004 Ritchie, K., Ropacki, M., Albala, B., Harrison, J., Kaye, J., Kramer, J., et al. Libon, D. J., Preis, S. R., Beiser, A. S., Devine, S., Seshadri, S., Wolf, P. A., et al. (2017). Recommended cognitive outcomes in preclinical Alzheimer’s disease: (2015). Verbal memory and brain aging: An exploratory analysis of the role of Consensus statement from the European Prevention of Alzheimer’s Dementia error responses in the Framingham study. Am. J. Alzheimers. Dis. Other Demen. project. Alzheimer’s Dement. 13, 186–195. doi: 10.1016/j.jalz.2016.07.154 30, 622–628. doi: 10.1177/1533317515577184 Sánchez-Nieto, J. M., Villa, M. A., and Mendoza-Núñez, V. M. (2016). Lloret-Segura, S., Ferreres-Traver, A., Hernández-Baeza, A., and Tomás-Marco, Rendimiento en el test de aprendizajeauditivo verbal de Rey en una población I. (2014). El Análisis Factorial Exploratorio de los Ítems: una guíapráctica, de adultosmayores de México. Rev. Mexicana Neurociencia Julio Agosto 17, revisada y actualizada. Anales de Psicol. 30, 1151–1169. doi: 10.6018/analesps. 37–44. 30.3.199361 Savage, S., Hoefeijzers, S., Milton, F., Streatfield, C., Dewar, M., and Zeman, A. Lu, H., Ni, X., Fung, A. W. T., and Lam, L. C. W. (2018). Mapping the proxies of (2019). The evolution of accelerated long-term forgetting: Evidence from the memory and learning function in senior adults with high-performing, normal TIME study. Cortex 110, 16–36. doi: 10.1016/j.cortex.2017.09.007 aging and neurocognitive disorders. J. Alzheimer’s Dis. 64, 815–826. doi: 10. Schmidt, M. (1996). Rey Auditory and Verbal Learning Test: A handbook. Los 3233/JAD-180225 Angeles, CA: Western Psychological Services. Mameniskiene, R., Jatuzis, D., Kaubrys, G., and Budrys, V. (2006). The Schoenberg, M. R., Dawson, K. A., Duff, K., Patton, D., Scott, J. G., and Adams, decay of memory between delayed and long-term recall in patients with R. L. (2006). Test performance and classification statistics for the Rey Auditory temporal lobe epilepsy. Epilepsy Behav. 8, 278–288. doi: 10.1016/j.yebeh.2005. Verbal Learning Test in selected clinical samples. Arch. Clin. Neuropsychol. 21, 11.003 693–703. doi: 10.1016/j.acn.2006.06.010 Marchand, D. G., Montplaisir, J., Postuma, R. B., Rahayel, S., and Gagnon, J. F. Shimizu, H. (1996). Rehearsal Strategies, Test Expectancy, and Memory (2017). Detecting the cognitive prodrome of dementia with lewy bodies: A Monitoring in Free Recall. Memory 4, 265–288. doi: 10.1080/096582196388951 Frontiers in Aging Neuroscience | www.frontiersin.org 14 February 2022 | Volume 14 | Article 809019 Alviarez-Schulze et al. RAVLT Validation and Long-Term Forgetting Speer, P., Wersching, H., Bruchmann, S., Bracht, D., Stehling, C., Thielsch, M., Wang, X., Shen, D., and Huang, H. (2016). Prediction of Memory Impairment with et al. (2014). Age- and gender-adjusted normative data for the German version MRI Data: A Longitudinal Study of Alzheimer’s Disease. Med. Image Comput. of Reys Auditory Verbal Learning Test from healthy subjects aged between 50 Comput. Assisted Intervent. 9900, 273–281. doi: 10.1007/978-3-319-46720-7_32 and 70 years. J. Clin. Exp. Neuropsychol. 36, 32–42. doi: 10.1080/13803395.2013. Wearn, A. R., Saunders-Jennings, E., Nurdal, V., Hadley, E., Knight, M. J., Newson, 863834 M., et al. (2020). Accelerated long-term forgetting in healthy older adults Squire, L. R. (1982). The neuropsychology of human memory. Annu. Rev. Neurosci. predicts cognitive decline over 1 year. Alzheimer’s Res. Ther. 12, 1–9. doi: 10. 5, 241–273. doi: 10.1146/annurev.ne.05.030182.001325 1186/s13195-020-00693-4 Szpunar, K. K., McDermott, K. B., and Roediger, H. L. (2007). Expectation of a Wechsler, D. (2012). WAIS-IV. Escala de inteligencia de Wechsler para adultos-IV. final cumulative test enhances long-term retention. Mem. Cogn. 35, 1007–1013. Manual de aplicación y corrección. Madrid: NCS Pearson, Inc. doi: 10.3758/BF03193473 Weitzner, D. S., Pugh, E. A., Calamia, M., and Roye, S. (2020). Examining the Then, F. S., Luck, T., Angermeyer, M. C., and Riedel-Heller, S. G. (2016). Education factor structure of the Rey auditory verbal learning test in individuals across as protector against dementia, but what exactly do we mean by education? Age the life span. J. Clin. Exp. Neuropsychol. 42, 406–414. doi: 10.1080/13803395. Ageing 45, 523–528. doi: 10.1093/ageing/afw049 2020.1741517 Thomas, K. R., Eppig, J., Edmonds, E. C., Jacobs, D. M., Libon, D. J., Au, R., Weston, P. S. J., Nicholas, J. M., Henley, S. M. D., Liang, Y., Macpherson, et al. (2018). Word-list intrusion errors predict progression to mild cognitive K., Donnachie, E., et al. (2018). Accelerated long-term forgetting in impairment. Neuropsychology 32, 235–245. doi: 10.1037/neu0000413 presymptomatic autosomal dominant Alzheimer’s disease: a cross-sectional Tort-Merino, A., Laine, M., Valech, N., Olives, J., León, M., Ecay-Torres, M., et al. study. Lancet Neurol. 17:123. doi: 10.1016/S1474-4422(17)30434-9 (2021a). Accelerated long-term forgetting over three months in asymptomatic Wyble, B., and Chen, H. (2017). Memory consolidation of attended information is APOE ε4 carriers. Ann. Clin. Transl. Neurol. 8, 477–484. doi: 10.1002/acn3. optional: Comment on Jiang et al. (2016). J. Exp. Psychol. Learn. Mem. Cogn. 43, 51245 997–1000. doi: 10.1037/xlm0000333 Tort-Merino, A., Valech, N., Laine, M., Olives, J., León, M., Ecay-Torres, M., Zhang, J., Zhou, W., Wang, L., and Zhang, X. (2017). Gender differences of et al. (2021b). Accelerated long-term forgetting in individuals with subjective neuropsychological profiles in cognitively normal older people without amyloid cognitive decline and amyloid-β positivity. Int. J. Geriatr. Psychiatry 36, 1037– pathology. Compr. Psychiatry 75, 22–26. doi: 10.1016/j.comppsych.2017.02.008 1049. doi: 10.1002/gps.5539 Zimmermann, J. F., and Butler, C. R. (2018). Accelerated long-term forgetting Tort-Merino, A., Valech, N., Peñaloza, C., Grönholm-Nyman, P., León, M., Olives, in asymptomatic APOE ε4 carriers. Lancet Neurol. 17, 394–395. doi: 10.1016/ J., et al. (2017). Early Detection of Learning Difficulties when Confronted with S1474-4422(18)30078-4 Novel Information in Preclinical Alzheimer’s Disease Stage 1. J. Alzheimer’s Dis. 58, 855–870. doi: 10.3233/JAD-161173 Conflict of Interest: The authors declare that the research was conducted in the Tulving, E. (2002). Episodic memory: from mind to brain. Annu. Rev. Psychol. 53, absence of any commercial or financial relationships that could be construed as a 1–25. doi: 10.1146/annurev.psych.53.100901.135114 potential conflict of interest. Vakil, E., and Blachstein, H. (1993). Rey Auditory-Verbal Learning Test: Structure analysis. J. Clin. Psychol. 49, 883–890. Publisher’s Note: All claims expressed in this article are solely those of the authors Vakil, E., Greenstein, Y., and Blachstein, H. (2010). Normative data for and do not necessarily represent those of their affiliated organizations, or those of composite scores for children and adults derived from the rey auditory verbal the publisher, the editors and the reviewers. Any product that may be evaluated in learning test. Clin. Neuropsychol. 24, 662–677. doi: 10.1080/138540409034 this article, or claim that may be made by its manufacturer, is not guaranteed or 93522 endorsed by the publisher. Vaqué-Alcázar, L., Mulet-Pons, L., Abellaneda-Pérez, K., Solé-Padullés, C., Cabello- Toscano, M., Macià, D., et al. (2021). tDCS-Induced Memory Reconsolidation Copyright © 2022 Alviarez-Schulze, Cattaneo, Pachón-García, Solana-Sánchez, Effects and Its Associations With Structural and Functional MRI Substrates in Tormos, Pascual-Leone and Bartrés-Faz. This is an open-access article distributed Subjective Cognitive Decline. Front. Aging Neurosci. 13:695232. doi: 10.3389/ under the terms of the Creative Commons Attribution License (CC BY). The use, fnagi.2021.695232 distribution or reproduction in other forums is permitted, provided the original Wamsley, E. J., Hamilton, K., Graveline, Y., Manceor, S., and Parr, E. (2016). Test author(s) and the copyright owner(s) are credited and that the original publication expectation enhances memory consolidation across both sleep and wake. PLoS in this journal is cited, in accordance with accepted academic practice. No use, One 11:1–15. doi: 10.1371/journal.pone.0165141 distribution or reproduction is permitted which does not comply with these terms. Frontiers in Aging Neuroscience | www.frontiersin.org 15 February 2022 | Volume 14 | Article 809019