education sciences Article The Emotional Competence Assessment Questionnaire (ECAQ) for Children Aged from 3 to 5 Years: Validity and Reliability Evidence Montse Bartroli 1,2 , Ariadna Angulo-Brunet 3,4, * , Marina Bosque-Prous 3,5 , Catrina Clotas 1 and Albert Espelt 3,6,7 1 Agència de Salut Pública de Barcelona, Pl Lesseps, 1, 08023 Barcelona, Spain; mbartrol@aspb.cat (M.B.); cclotas@aspb.cat (C.C.) 2 Departament de Ciències Experimentals i de la Salut, Universitat Pompeu Fabra, C/Dr. Aiguader, 88, 08003 Barcelona, Spain 3 Departament de Psicobiologia i de Metodologia de les Ciències de la Salut, Universitat Autònoma de Barcelona (UAB), C/ Fortuna s/n, 08193 Bellaterra (Cerdanyola del Valles), Spain; mbosquep@uoc.edu (M.B.-P.); aespelt@umanresa.cat (A.E.) 4 Faculty of Psychology and Education Sciences, Universitat Oberta de Catalunya, Rbla. Poblenou, 156, 08018 Barcelona, Spain 5 Faculty of Health Sciences, Universitat Oberta de Catalunya, Rbla. Poblenou, 156, 08018 Barcelona, Spain 6 Facultat de Ciències de la Salut de Manresa. Universitat de Vic—Universitat Central de Catalunya (UVicUCC), Av. Universitària, 46, 08242 Manresa, Spain 7 Centro de Investigación Biomédica en Red de Epidemiología y Salud Pública (CIBERESP), C/ Monforte de Lemos 3 Pabellón 11, 28029 Madrid, Spain * Correspondence: aangulob@uoc.edu Citation: Bartroli, M.; Angulo-Brunet, Abstract: In order to assess emotional competence in children, it is necessary to have psychometrically A.; Bosque-Prous, M.; Clotas, C.; sound measures. To the best of our knowledge, there is no available tool to assess emotional Espelt, A. The Emotional competence in children from 3 to 5 years old that assesses the five emotional competences of the Competence Assessment Bisquerra model and can be easily and quickly answered in the school environment. The objective of Questionnaire (ECAQ) for Children this study is to develop a measure, the Emotional Competence Assessment Questionnaire (ECAQ), Aged from 3 to 5 Years: Validity and and to provide evidence of its psychometric quality. Qualitative evidence was obtained from a Reliability Evidence. Educ. Sci. 2022, systematic review, from two expert committees and from five discussion groups. On the other 12, 489. https://doi.org/10.3390/ hand, quantitative validity and reliability evidence was obtained from a sample of 1088 students educsci12070489 and other smaller subsamples. The results suggest that the ECAQ is a short and easy-to-use tool, Academic Editor: Colleen easily understood by administrators. The quantitative results confirm a general factor of emotional McLaughlin competence adjusted for three specific factors. This factor has excellent internal consistency and Received: 1 April 2022 test-retest reliability. The ECAQ has therefore been shown to be a promising tool for assessing Accepted: 12 July 2022 emotional competence in children between 3 and 5 years of age. Published: 15 July 2022 Keywords: validity; reliability; assessment; emotional competence; children Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations. 1. Introduction According to the World Health Organization, health, and particularly mental health, can be promoted by boosting emotional competence in children [1]. In addition, some Copyright: © 2022 by the authors. studies point to the key role of emotional competence in influencing the severity of long- Licensee MDPI, Basel, Switzerland. term mental health problems [2]. Emotional competence can be enhanced in the school This article is an open access article environment through structured programs of emotional and social education [3]. The distributed under the terms and school is not only the place where children spend a good part of their day but also the conditions of the Creative Commons place where equitable access to programs to increase different competences, including Attribution (CC BY) license (https:// emotional competence, can be guaranteed. The implementation of these programs has creativecommons.org/licenses/by/ proven to be useful for improving academic performance [4], relationships with peers 4.0/). Educ. Sci. 2022, 12, 489. https://doi.org/10.3390/educsci12070489 https://www.mdpi.com/journal/education Educ. Sci. 2022, 12, 489 2 of 19 and adults, problem solving, self-control, perceived self-efficacy, school involvement and well-being [2,5,6]. Emotional competence has also been related to executive function, the latter being below average in children with low emotional competence [7]. Different school- based universal interventions and programs for children aim to enhance their emotional competence [3], but their outcomes should always be evaluated to ensure resources are allocated to interventions with proven positive results [8]. Therefore, we need tools that help us to assess emotional competence in a population-based way. In an educational context, there are different methods to measure emotional compe- tence in children. When children can read and write properly, we can use self-administered tests. In children aged from 3 to 5 years, although there are studies that suggest the possibil- ity of self-assessment using photographs or drawings, other forms are required, especially when many children are to be assessed [9]. Some procedures are qualitative and include direct observation of the children when they are asked to do a specific activity [10–12]. Other procedures are quantitative, and the diversity of them is higher. Moreover, some tests are administered to children by trained researchers and others are answered by teach- ers [13–15] or parents [16,17]. In public health, it is easier to access teachers than families, as teachers are responsible for implementing and evaluating different school-based universal interventions and programs. Most methods assess just one emotional competence, usually emotional regulation, as it occurs for the Emotional Regulation Checklist (ERC), a 24-item scale used in different investigations [18,19]. If they evaluate more than one competence, the number of items is usually high, as for the Behaviors Underpinning Skills for Social- Emotional School Readiness (BUSSE-SR), a social-emotional questionnaire that assesses self-awareness, self-regulation, social communication, empathy and adaptive behavior [20], or the ACER Well-being Survey (teacher form) that assesses social-emotional well-being and social-emotional competence, each containing 50 items [21]. After an extensive literature review, Bisquerra and his research group in emotional education from the University of Barcelona (Grup de Recerca i Orientació Psicopedagògica— GROP), who have trained teachers in emotional education since 2000, established that emotional competence includes five emotional competences [22]. Three of them are of an intrapersonal nature: (i) emotional awareness—the ability to know what you feel and what others feel and why—(ii) emotional regulation—the ability to regulate unpleasant emotions and enhance pleasant ones—and (iii) emotional autonomy—an ability related to self-esteem and the strength to be self-motivated and maintain a positive attitude in spite of adversity. Moreover, the other two are of an interpersonal nature: (iv) social competence—the ability to establish and maintain positive relationships with others—and (v) life skills—related to the ability to adopt appropriate problem-solving. The last three are associated with the capacity to generate positive life experiences and well-being. Each of the competences included in this model overlap to some extent, forming part of the same construct of emotional competence. This model is inspired on Goleman’s emotional intelligence model, which includes five competences—emotional self-awareness, emotion management, self- motivation, empathy and social skills, which in turn include the existence of twenty-five competences [23]. The model proposed by Bisquerra includes problem solving as a main skill and its structure facilitates the understanding of the broad construct of emotional competence defined as “the set of knowledge, abilities, skills, and attitudes necessary to understand, express, and appropriately regulate emotional phenomena” [22]. It is also the model under which most of the teachers in Barcelona are trained. To the best of our knowledge, no test evaluates the five competences mentioned above and at the same time can be answered easily and quickly. The latter aspect is important from a public health perspective, as there is the need to evaluate a large number of children to design population interventions. We propose to fill this gap by developing a new test, the Emotional Competence Assessment Questionnaire (ECAQ), which is answered by teachers to evaluate emotional competence considering the five emotional competences in children aged from 3 to 5 years. In this process, we follow the Standards for Educational and Psycho- logical Testing [24], one of the most well-known guidelines related to the development and Educ. Sci. 2022, 12, 489 3 of 19 interpretation of tests, which are used to provide evidence of their psychometric quality. The Standards state that “validity refers to the degree to which evidence and theory sup- port the interpretations of test scores for proposed uses of tests” [24]. To provide evidence of validity, these guidelines suggest five sources of validity evidence: content, response processes, the consequences of testing, internal structure and relations to other variables. To analyze the reliability of a questionnaire, its internal consistency can be assessed as well as its replicability by test-retesting. The hypotheses of our study are: (a) the ECAQ will show good evidence of validity and reliability; (b) when examining the relationship with other variables, we will find a high correlation between the ECAQ total score and other scales or subscales measuring emotional regulation or social competence (i.e., the Student Rating Scale [25], the Social Competence Scale—Teacher [26] and the subscale the Emotional Regulation Checklist [27]) and a low correlation between the ECAQ and other scales or subscales measuring emotional lability (i.e., the subscale the Lability and Negativity of the Emotional Regulation Checklist) and (c) in the assessment of test dimensionality, we will find a general factor including all five competences. Therefore, the general objective of this study is to develop and provide evidence of the psychometric quality of the ECAQ. To do so, we first propose to provide qualitative evidence by gathering validity evidence based on test content, response processes and test outcomes. We also propose to provide quantitative evidence by gathering validity evidence based on internal structure and relationships with other variables and by assessing the internal consistency reliability of the ECAQ and its test-retest reliability. 2. Materials and Methods 2.1. Participants 2.1.1. Qualitative Evidence Stage A multidisciplinary committee of seven experts in emotional education were invited to assess the first generated questionnaire’s items. Selected experts were professors (86% female) of postgraduate emotional education in Spanish universities. Another committee of four teachers (100% female) were invited to assess the items proposed by the experts committee. All agreed to participate when invited. Finally, we organized five discussion groups with a maximum of 10 teachers. To be eligible, teachers had to be actively teaching children three to five years old. 2.1.2. Quantitative Evidence Stage Eleven schools in Barcelona were eligible to collaborate and all of them agreed to take part in the study. A total of 1088 children (48.1% female) aged three to five years old were included in this part of the study once their teachers (n = 52) had answered the ECAQ. To assess validity evidence based on relations to other variables, these same teachers were also asked to respond to other questionnaires for the first 8 students on their class rosters. We obtained responses from 51 teachers. One teacher did not respond because he left school due to illness. Thus, 134 responses were obtained from 21 teachers for the ERC (46.2% girls), 119 from 17 teachers for the SRS (56.3% girls) and 89 from 13 teachers for the SCS-T (44.9% girls). To assess test-retest reliability, we asked six teachers from two different schools to answer the ECAQ twice for each child in their class. A total of 121 responses were included in this part of the study. 2.2. Measures The Emotional Competence Assessment Questionnaire (ECAQ) is a questionnaire assessing the emotional competence of children from 3 to 5 years of age. The ECAQ has a total of 30 items and uses a Likert-type scale with six response categories (1 = "never” to 6 = “always”). The ECAQ was developed in Catalan and Spanish and was only adapted Educ. Sci. 2022, 12, 489 4 of 19 to English for publication purposes. It is based on the emotional competence model of Bisquerra [22]. Respondents are the children’s teachers. The Emotion Regulation Checklist (ERC) [27] is a scale assessing the emotion reg- ulation of children up to 12 years of age with 24 items in two subscales (Lability and Negativity and Emotional Regulation). It can also be used in a unidimensional scale using a 4-point Likert scale (1 = “never” to 4 = “almost always”). Our study used the Spanish [28] and Catalan (Cicchetti D, personal communication) versions of the ERC. It has not been necessary to adapt the instrument to the application context. We have obtained for the total score a Cronbach’s alpha of 0.79. This instrument includes questions such as the following: “Its easily frustrated” (Lability and Negativity Subscale) and “Can say when s/he is feeling sad, angry or mad, fearful or afraid” (Emotional Regulation Subscale). The Student Rating Scale (SRS) [25] is 33-item scale in a 7-point Likert scale (1 = “not at all like this child” to 7 = “very much like this child”) assessing 11 different domains related to social-emotional learning and health promotion in children from 3 years of age. The domains are: (1) understanding of positive action, (2) self-concept, (3) physical health, (4) intellectual health, (5) self-management, (6) self-control, (7) respect, (8) considerate, (9) social bonding, (10) honesty and (11) self-improvement. In this research, we used a total score. This scale was completed by adult observers familiar with the child. Two researchers not involved in this research translated the questionnaire into Spanish and Catalan, separately. A third researcher resolved differences in translation. In this sample, the internal consistency was excellent (Cronbach’s alpha = 0.97, omega = 0.92). This instru- ment includes questions such as the following: “Is generally happy, outgoing, optimistic, confident, feels good about him/herself” (self-concept domain), “Shows self-control” (self- control domain) and “Persist in tasks, turns problems into challenges, receives suggestions” (self-improvement domain). The Social Competence Scale—Teacher (SCS-T) [26] is a 25-item scale in a 5-point Likert scale (0 = “not at” to 4 = "very well”) assessing prosocial behaviors, emotional self-regulation and academic skills in children from 3 years of age. Each item states a behavior that a child may display at school. Respondents are the children’s teachers. The scale contains three subscales: prosocial/communication skills, emotional regulation skills and academic skills. The SCS-T can also be used on a one-dimensional scale assessing social competence and a total score can be calculated. Two researchers not involved in this research translated the questionnaire into Spanish and Catalan, separately. A third researcher resolved differences in translation. Inverse items were recoded so that a high score would indicate high possession in that dimension. We obtained for the total score a Cronbach’s alpha of 0.97. This instrument includes questions such as the following: “Understands others” (prosocial/Communication skills dimension), “Can calm down” (emotional Regulation Skills dimension) and “Is a self-starter” (academic skills dimension). 2.3. Procedure and Data Analysis 2.3.1. Qualitative Evidence Stage The procedure for obtaining evidence of validity based on test content involved three key steps: the development of the items, their assessment by a committee of emotional education experts and the gathering of teachers’ opinions. In the first two steps, three elements were considered: the definition, representation and relevance of the domain [29]. The conceptual definition of the assessment questionnaire and the initial preparation of the items were derived from scientific literature (theoretical basis) and involved three professional researchers with expertise in this area. In January 2017, we queried online databases, such as PubMed, Embase, PsycINFO and the Cochrane Database of Systematic Reviews, for methods used to assess at least one of the five emotional competences in 3- to 5-year-old children. The search strategy can be seen in Table 1. To be eligible, tests needed to meet the 5 criteria listed in Table 2. The researchers reviewed the items of the selected tests and chose the ones that best fit the conceptual definition of the ECAQ questionnaire. They also brainstormed on missing content and created new items. Educ. Sci. 2022, 12, 489 5 of 19 Table 1. Search strategy for methods used to assess at least one of the five emotional competences in children aged from 3 to 5 years. January 2017. Database Search Strategy (“emotional intelligence” [All Fields] OR “emotional education” [All Fields] OR “emotion regulation” [All Fields] OR “emotion recognition” [All Fields] OR “emotion competence” [All Fields] OR “emotional awareness” [All Fields] OR “emotional regulation” [All Fields] OR “emotional learning” [All Fields] OR “emotional literacy” [All Fields] OR “socioemotional skills” [All Fields] OR “life skills” [All Fields] OR “emotional skills” [All Fields]) AND ((“evaluation studies” [Publication Type] OR “evaluation studies as topic” [MeSH Terms] OR “evaluation” [All Fields]) OR (“Assessment” [Journal] OR “assessment” [All Fields]) OR program[All Fields] OR (“Intervention (Amstelveen)” [Journal] OR “intervention” [All Fields] OR “Interv Sch Clin” [Journal] OR “intervention” [All Fields]) OR (“research design” [MeSH Terms] OR (“research” [All Fields] AND “design” [All Fields]) OR “research design” [All Fields] OR “test” [All Fields]) OR (“weights and measures” [MeSH Terms] OR (“weights” [All Fields] AND PubMed “measures” [All Fields]) OR “weights and measures” [All Fields] OR “scale” [All Fields]) OR tool[All Fields] OR (“Measurement (Lond)” [Journal] OR “measurement” [All Fields] OR “Measurement ( Mahwah N J)” [Journal] OR “measurement” [All Fields]) OR (“weights and measures” [MeSH Terms] OR (“weights” [All Fields] AND “measures” [All Fields]) OR “weights and measures” [All Fields] OR “measure” [All Fields]) OR instrument[All Fields] OR (“indicators and reagents” [Pharmacological Action] OR “indicators and reagents” [MeSH Terms] OR (“indicators” [All Fields] AND “reagents” [All Fields]) OR “indicators and reagents” [All Fields] OR “indicator” [All Fields])) AND ((“child” [MeSH Terms] OR “child” [All Fields] OR “children” [All Fields]) OR kids[All Fields] OR preschool[All Fields] OR (“child, preschool” [MeSH Terms] OR (“child” [All Fields] AND “preschool” [All Fields]) OR “preschool child” [All Fields] OR “preschooler” [All Fields])) (“emotional intelligence” OR “emotional education” OR “emotion regulation” OR “emotion recognition” OR “emotion learning” OR “emotion competence” OR “emotional awareness” OR “emotional regulation” OR “emotional learning” Embase OR “emotional literacy” OR “socioemotional skills” OR “life skills” OR “emotional skills”) AND (evaluation OR assessment OR program OR intervention OR test OR scale OR tool OR measurement OR measure OR instrument OR indicator) AND (children OR kids OR preschool OR preschooler) (“emotional intelligence” OR “emotional education” OR “emotion regulation” OR “emotion recognition” OR “emotion learning” OR “emotion competence” OR “emotional awareness” OR “emotional regulation” OR “emotional learning” PsycINFO: OR “emotional literacy” OR “socioemotional skills” OR “life skills” OR “emotional skills”) AND (evaluation OR assessment OR program OR intervention OR test OR scale OR tool OR measurement OR measure OR instrument OR indicator) AND (children OR kids OR preschool OR preschooler) Cochrane Database of http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/ Systematic (accessed on 15 January 2017) Reviews Table 2. Required criteria for tests to be eligible for selection. January 2017. Criteria Description of the Criteria Criterion 1 At least one emotional competence was assessed. Criterion 2 Free access to the test was authorized by the author/s. Criterion 3 The respondents were teachers. Criterion 4 Evidence of reliability and validity was provided. Criterion 5 The test was used in at least one research project. Educ. Sci. 2022, 12, 489 6 of 19 After the three researchers had selected and created the items, the multidisciplinary committee of seven experts evaluated them. We used the conceptual framework of the Delphi method [30]. In the first round, we sent the experts an initial list of items and a definition of each emotional competence we intended to evaluate with the questionnaire [31]. We asked them to classify each item into one of the five emotional competences. They also evaluated whether the items were appropriate (“Please specify whether each item is appropriate. An appropriate item should be well described and relevant to the competence where you have previously classified it”). In the case of inappropriate items, we asked the experts whether they would delete them or suggest modifications to improve them. We selected the items that all experts, or all but one, classified in the same emotional competence and redesigned the ones which most experts had said were not well built. This way, a second version of the questionnaire was obtained. In the second round, we sent the seven experts the new version of the questionnaire. For each item, we asked if it was classified in the correct emotional skill and, if not, to indicate the reason. Then, we revised the given reasons. The experts also evaluated whether there were any similarities between items. In this case, we asked them to choose which items should be part of the questionnaire. Finally, we asked which ones the experts would eliminate, as the final version of the questionnaire should have had no more than 30 items. This limit was set considering that the teachers could spend a maximum of 10 min per child by answering 3 questions per minute. The experts also had the opportunity to suggest new items. Moreover, if at least half of them considered that two or more items were similar, we kept the one selected by the majority. Lastly, we deleted those that all or all but one expert said that should be eliminated. Then, we sent this version to four teachers to gather their opinions before sending it again to the experts. Therefore, a third version of the questionnaire was obtained. In the third and final round, we sent the experts the third version of the questionnaire. We asked them what percentage of items each competence should have. From the responses, we calculated the average of the percentages of items that each expert indicated for a competence. We also asked the experts to give a score of 1 to 4 to each item to indicate its relevance for the skill to which it was attributed (1 = “not truly relevant”; 4 = “truly relevant”). Finally, we selected the items that had the highest average score and obtained a fourth version of the questionnaire. At the end of the second round with experts, the resulting items were sent via e-mail to the teachers’ committee. We encouraged these teachers to suggest possible modifications or propose relevant aspects that should be considered but had not been included in the test. For each item, the teachers answered the question “what do you think is evaluated with this item?” They also indicated if they could answer referring to their children. In the case of a negative answer, they were encouraged to make a new proposal. Finally, they also had the opportunity to recommend new items or suggest modifications to the existing ones. If at least two out of four did not correctly explain what an item was intended to evaluate, or affirmed that they could not evaluate it, the item was removed. To gather validity evidence based on the response processes, we tried to make relevant adjustments according to the performance and responses of the teachers’ committee. Inter- views were sent to them by e-mail. First, teachers were asked to answer the questionnaire (the fourth version) about five children in their class. Once this phase was completed, the fifth version of the test was obtained. To obtain evidence of validity based on the consequences of testing, we asked 52 teach- ers from 11 different schools in the city of Barcelona to answer the ECAQ for each child in their class (25 on average), at the beginning and at the end of the school year. To evaluate if there were any consequences that could bias the results obtained, we conducted five discussion groups with a maximum of 10 teachers in each one of them. Two questions were formulated in these groups: “Did you feel that your work was under evaluation and that you should change the score of a child if they got a low score in the test? In case you Educ. Sci. 2022, 12, 489 7 of 19 respond yes, in what way did you feel that your work was under evaluation?” and “Do you think that the fact of having to answer the test twice somehow influenced you in scoring the children? If yes, how do you think it influenced you?” 2.3.2. Quantitative Evidence Stage The teachers (n = 52) did not receive any training before answering the ECAQ, but they received an official accreditation for the hours spent answering questionnaires. We then invited the teachers also to answer other questionnaires for the first 8 students on their class rosters because the task was very demanding and required a considerable effort in time. A sample of 342 children was finally obtained. To study the internal structure of the questionnaire, we assessed four confirmatory factor analyses models that were compatible with theory with the R package lavaan [32]: a five-factor correlated model; a three-factor correlated model; a unidimensional model and a general factor with all items and three specific theoretical components (i.e., awareness, internal/external regulation and wellness). Considering that items had six response op- tions and skewness and kurtosis values were above 1 point (see Supplementary Material; Figure S1), we used the unweighted least squares estimator and the Pearson’s correlation as input matrixes. To assess the goodness of fit indexes (GOFIs), we considered as excellent fit a value with a Tucker–Lewis index (TLI) and comparative fit index (CFI) above 0.95 and with a root mean square error of approximation (RMSEA) below 0.05 [33]. Alternatively, we considered as acceptable an RMSEA value below 0.08 [34]. To examine the relationship with other variables, we performed Pearson correlations: the ECAQ total score was correlated with other tools that measure similar constructs (i.e., SCS-T and SRS) or different constructs (i.e., ERC subscale Lability and Negativity). To assess the ECAQ’s reliability, we evaluated internal consistency using the coefficient omega for the general factor with all items and three specific theoretical components (i.e., awareness, internal/external regulation and wellness) [35,36]. We also obtained Cronbach’s alpha to compare it with previous results. Furthermore, as recommended by Rodriguez et al. (2016) [35], we also computed the Explained Common Variance for the general factor. We considered values above 0.70 as acceptable [37]. We estimated the intraclass correlation coefficient (ICC) with the best values closest to 1.00. We chose absolute agreement ICC because it takes into account systematic differences as part of the measurement error, as there is no clear consensus regarding the cutoff point for the ICC. To assess the test-retest reliability, the teachers answered the ECAQ for each child on two separate occasions (time lapse of one month). Neither the teachers nor the children who participated in this assessment had participated in the validity evidence part of the study. During the second ECAQ round, the respondents did not have access to the responses given in the first round to prevent any form of recall bias. The schools that participated in the test-retest reliability evaluation received a collection of books on emotional education in appreciation for their collaboration in the study. 3. Results 3.1. Evidence of Validity Based on Test Content From our database search, we found 164 instruments used to assess at least one of the five emotional competences established by the GROP. After checking eligibility against our five criteria (Table 2), we selected various instruments from which we extracted 126 items that we considered best fit the model of the five emotional competences. We also created 12 new items to cover the aspects that were not addressed by the selected pre-existing ones. This way, we obtained an initial version of the questionnaire of 138 items. The development of the ECAQ can be found in Figure 1. In the first round of seven experts, 67 items were selected, as they were classified in the same emotional competence by all experts, or by all but one. Of these items, we reformulated the ones that were considered not to be well developed by all experts (n = 7). Educ. Sci. 2022, 12, 489 8 of 19 1st round: experts 138 initial items 71 items discarded 18 items classified in the 42 items classified in 7 items reformulated same competence by ALL the same competence experts by all but one expert 67 items remaining 2nd round: experts 59 items ALL experts 8 items at least one 22 items similar to agreed they were well expert said it was not others and not chosen classified in well classified. Not (7 of which all the 11 new items competences removed experts or all but one agreed to delete) 56 items remaining Teachers´s opinion 9 items incomprehensible 0 new items 47 items remaining 3rd round: experts Theoretical assigment of number of items by competence. Total number of items: 30 7 emotional awareness 7 emotional regulation 5 emotional autonomy 6 social competence 5 life skills Items relevance evaluation by competence 2 excluded (22%) 1 excluded (13%) 0 excluded 10 excluded (63%) 4 excluded (44%) Figure 1. Development of test items. In the second round, 59 items were considered to be well classified by all the experts. On the contrary, the other eight items were considered not to be properly classified by at least one expert. Nevertheless, most of the experts had valued these eight items as well classified and the arguments offered by those who disagreed were not congruent with our theoretical model. For example, related to the item classified in the emotional awareness competence “The child is aware that emotions are related to how we behave”, an expert said it should be classified in emotion regulation competence although in no case does it refer to regulation strategies, and so we considered it was focused on awareness of the impact of emotion on behavior. Therefore, we decided to maintain these items in the competence in which they were initially assigned. Finally, we deleted 22 items that the experts considered similar to others that were chosen to be included in the test. Moreover, an expert proposed 11 new items that were included in the questionnaire. Finally, 56 items were obtained at the end of this round. Educ. Sci. 2022, 12, 489 9 of 19 This version of the questionnaire was examined by four teachers. We removed nine items because at least two of the teachers did not correctly explain what the items were evaluating and/or they were not able to evaluate them. None of the teachers suggested new items; therefore, a test version of 47 was obtained. In the third round, we assigned a percentage of 22.5% to emotional awareness (seven items), 22.5% to emotional regulation (seven items), 18% to emotional autonomy (5 items), 21% to social competence (six items) and 16% to life skills (five items). We finally selected the items with the highest average scores and 17 were removed to obtain a maximum of 30 in a preliminary version of the test. 3.2. Evidence of Validity Based on Response Processes The average time teachers took to answer the test was 7 min per child (an average of 3 h for each class). Following a suggestion from teachers, we added the question “Does the child have any special educational needs?” We also changed the response categories to “never”, “rarely”, “sometimes”, “often”, “very often” and “always”, as they said that the suggested ones (“totally disagree”, “disagree”, “agree” and “totally agree”) made it difficult to answer the questionnaire. 3.3. Evidence of Validity Based on the Consequences of Testing A total of 36 teachers agreed to participate in the five discussion groups. Eighty-nine percent of them (32 out of 36) did not feel that their work was under evaluation and that they should change it in case a child got a low score. A total of 94% (34 out of 36) said that the fact of having to answer the test twice did not influence them in the task. 3.4. Evidence of Validity Based on the Internal Structure The GOFIs for all tested models are shown in Table 3. First, consistently with the preceding literature and the development of the tool, we tested two models: (1) a five- factor correlated model and (2) a three-factor correlated model. The GOFIs were above the acceptable limits for the RMSEA values (0.11 and 0.09, respectively). A detailed analysis of the results of the two models (for the complete output, see Supplementary Material) indicated a lack of discrimination between the questionnaire factors. The correlation between the factors ranged between 0.77 and 0.98 for the five-factor model and between 0.78 and 0.87 for the three-factor model. Subsequently, a one-dimensional solution was evaluated, also obtaining high RMSEA values. Considering the previous results and the fact that the questionnaire was developed by experts who contemplated evaluating three to five dimensions, we evaluated a bifactor model with a general factor and with factors specific to emotional competences. The model tested with five specific and one general factors did not converge (results not shown). On the other hand, the model with three specific and one general factor yielded suitable GOFIs. Table 3. Comparison of Fit Indexes in Models Fitted to Confirmatory Factor Analysis. RMSEA Model X2 df CFI TLI RMSEA 90%CI 5 factors 5990.54 395 0.98 0.98 0.11 [0.11–0.12] 3 factors 4296.12 402 0.99 0.99 0.09 [0.09–0.10] 1 factor 8737.37 405 0.98 0.97 0.14 [0.14–0.14] Bifactor 2357.29 375 0.99 0.99 0.07 [0.07–0.07] (3 specific) X2 : chi-square; df: degrees of freedom; CFI: comparative fit index; TLI: Tucker–Lewis Index; RMSEA: root mean square error of approximation. Figure 2 shows the results for the standardized solution of the bifactor model. Factor loadings were high for the general factor (range 0.60–0.95) but not for the three specific factors. The factor loadings of the awareness factor ranged between 0.00 and 0.52. The correspondence between the factors of the bifactor model and the item content can be found Educ. Sci. 2022, 12, 489 10 of 19 Educ. Sci. 2022, 12, x FOR PEER REVIEW in Table 11 and 4. Similarly, the factor loadings of the well-being factor ranged between 0.19 of 19 0.65. Finally, the factor loadings for the well-being factor ranged between 0.09 and 0.54. This analysis gives favorable evidence for the unidimensionality of the ECAQ if specific factors are considered for modeling the common variance of theoretical variance. Figure 2. Standardized factor loadings for the bifactor model of the Emotional Competences Assess- Figure 2. Standardized factor loadings for the bifactor model of the Emotional Competences Assess- ment mentQuestionnaire. Questionnaire. Table 4. Item content for the Emotional Competence Assessment Questionnaire (ECAQ) and factor loadings for general factors and for specific factors. Factor Load- Factor Load- Factor Item ings for Gen- ings for Spe- eral Factor cific Factors The child can express why he/she or another awa1 0.73 0.51 person can feel a certain emotion The child is able to communicate aspects of awa2 himself/herself of which to feel proud about, 0.73 0.48 both when asked and spontaneously The child identifies body signs (facial expres- awa3 sion, posture, and tone of voice) reflecting every 0.74 0.32 emotion Educ. Sci. 2022, 12, 489 11 of 19 Table 4. Item content for the Emotional Competence Assessment Questionnaire (ECAQ) and factor loadings for general factors and for specific factors. Factor Loadings for Factor Loadings for Factor Item General Factor Specific Factors The child can express why he/she or awa1 another person can feel a 0.73 0.51 certain emotion The child is able to communicate aspects of himself/herself of which to feel awa2 0.73 0.48 proud about, both when asked and spontaneously The child identifies body signs (facial awa3 expression, posture, and tone of voice) 0.74 0.32 reflecting every emotion The child can explain what emotion awa4 0.75 0.52 he/she is feeling The child knows that different people awa5 can feel different emotions when facing 0.82 0.13 the same situation The child is aware that what he/she awa6 0.83 0.05 does can affect how others feel The child recognizes that emotions can awa7 0.95 0.07 be preceded by a specific situation The child is able to make coherent decisions considering how he/she feels, awa8 0.88 0.00 what he/she needs, and the impact that the decision will have The child is aware that his/her emotions awa9 0.89 0.02 influence subsequent behaviors The child controls his rage when there is reg1 0.65 0.57 a disagreement reg2 The child waits his/her turn 0.49 0.62 The child can autonomously control reg3 his/her excitement not to bother 0.60 0.65 the others The child manages his/her negative reg4 emotions independently, not lingering 0.75 0.36 on them The child uses appropriate strategies to reg5 0.77 0.39 control anxiety When the child has a problem, he/she reg6 0.79 0.24 strives to see the positive side of things reg7 The child demonstrates self-control 0.68 0.59 The child treats the others fairly and reg8 0.71 0.42 tolerates differences reg9 The child is able to share things 0.63 0.50 The child can soothe himself/herself reg10 0.67 0.57 when excited or angry The child is able to accept and tolerate reg11 that things do not turn out as 0.68 0.53 he/she expected The child can do an activity for a while, enjoying it and concentrating while reg12 0.74 0.19 being oblivious to what happens elsewhere wel1 The child usually has a positive attitude 0.76 0.21 wel2 The child helps the others 0.76 0.09 Educ. Sci. 2022, 12, 489 12 of 19 Table 4. Cont. Factor Loadings for Factor Loadings for Factor Item General Factor Specific Factors The child is able to initiate social wel3 0.71 0.40 contacts and interactions wel4 The child seems to be emotionally well 0.72 0.42 wel5 The child is able to enjoy 0.74 0.50 wel6 The child has a positive self-image 0.71 0.54 The child has reasonable wel7 self-expectations considering his/her 0.83 0.20 age and the period of life The child feels confident and wel8 0.72 0.42 self-confident The child shows good social skills wel9 0.76 0.38 with friends Awa: awareness, reg: regulation, wel: wellness. 3.5. Evidence of Validity Based on Relations to Other Variables Table 5 shows descriptive statistics and correlations between the main variables. As can be seen, the children obtain scores above the mid-point on the ECAQ. The same occurs with the scores of the ERC, SRS and SCS-T. Therefore, the children in the sample show good emotional competence scores in both the ECAQ questionnaire and the other questionnaires used. Table 5. Descriptive Statistics and Convergent Validity Evidence. M SD Min Max ECAQ 1 2 3 4 5 6 7 ERC Subsample (n = 134) ECAQ 117.69 27.44 49 177 1.00 Total ERC (1) 50.49 8.13 27 78 −0.29 1.00 Lability/Negativity (2) 23.28 7.30 14 55 −0.62 0.84 1.00 Regulation (3) 23.27 4.46 11 32 0.63 0.22 −0.33 1.00 SRS Subsample (n = 119) ECAQ 126.73 27.87 39 178 1.00 Total SRS 172.18 31.29 62 219 0.79 SCS-T Subsample (n = 89) ECAQ 123.75 29.08 63 179 1.00 Total SCS-T (4) 82.16 18.58 43 119 0.83 1.00 Prosocial (5) 25.39 6.00 13 37 0.83 0.95 1.00 Regulation(6) 32.75 7.73 19 48 0.80 0.95 0.88 1.00 Academic (7) 24.01 5.96 10 34 0.70 0.92 0.83 0.80 1.00 ECAQ: Emotional Competences Assessment Questionnaire; ERC: Emotion Regulation Checklist; SRS: Student Rating Scale; SCS-T: Social Competence Scale—Teacher; M: mean; Min: minimum value; Max: maximum value; SD: standard deviation. To obtain evidence of the convergent and discriminant validity of the ECAQ, the correlation of the total scale score was evaluated against that of other questionnaires (ERC, SRS and SCS-T). As expected, the total score of the ECAQ showed a high negative correlation with the subscale Lability and Negativity (−0.62). This shows that the higher the emotional competence (measured with the ECAQ) which includes the ability to regulate negative moods and enhance positive ones, the lower the emotional instability and tendency for negativity. We also found a high positive correlation with the subscale Emotional Regulation (r = 0.63), showing that greater emotional competence (measured with the ECAQ), which includes the ability to manage emotions, correlates positively with the subscale measuring emotional regulation. We found high correlations with the SRS (r = 0.79) and the SCS-T (r= 0.83) and with the SCS-T subscales (range: 0.70–0.83), showing that when we obtain high scores in emotional competence measured with the ECAQ, we also obtain Educ. Sci. 2022, 12, 489 13 of 19 them when measuring socioemotional learning and social competence measured with the SRS and the SCS-T. Therefore, we can say that we have positive evidence of convergent and discriminant validity. 3.6. Reliability Regarding internal consistency reliability, a hierarchical omega of 0.90 for the general factor and a Cronbach’s alpha of 0.95 were obtained, which were considered excellent. Furthermore, the explained common variance results showed that 77% of the variance was accountable to the general factor. Figure 3 shows the test and retest scores and the linear regression line (y = 40.74 + 0.62 × Educ. Sci. 2022, 12, x FOR PEER REVIEW test). To assess the absolute agreement between both scores, the test-retest reliability 14 of 19 was evaluated with the ICC. The correlation obtained was 0.78 and the ICC was 0.76 (95% Cl = 0.67–0.83), which can be considered good. Figure 3. Figure Scatter plot 3. Scatter plot of of test test and and retest retest scores scores of of ECAQ ECAQ and and trend linear line. regression line. 4. Discussion 4. Discussion This study develops and provides evidence of the validity and reliability of the ECAQ. This study The ECAQ develops is based and provides on Bisquerra’s evidence theoretical modelofofthe validitycompetence, emotional and reliability whichof the dis- ECAQ. The ECAQ is based on Bisquerra’s theoretical model of emotional tinguishes, as recommended in the literature, between the intrapersonal and interpersonal competence, which domains distinguishes, in emotional ascompetence. recommended Thein process the literature, between the of development hasintrapersonal shown positive and evi- in- terpersonal dence of thedomains in emotional test content, competence. response process The process of and consequences oftesting. development On the hasothershown hand, positive evidence of the test content, response process and consequences the provided quantitative evidence of psychometric quality supports using a general score of testing. On the other hand, the provided quantitative evidence of psychometric quality to measure the emotional competence of children three to five years old. Moreover, results supports us- ing a general score to measure the emotional competence of children show positive evidence of convergent and discriminant validity and excellent reliability three to five years old. Moreover, results show positive evidence of convergent and discriminant validity indicators. and excellent reliabilityofindicators. The development this test has not only been a result of a bibliographic review from The development Bisquerra’s theoreticalof this test model buthas alsonotofonly been a result of the participation of aexperts bibliographic and thereview opinionsfromof Bisquerra’s theoretical Moreover, the target population. model butthis alsoisofaligned the participation of experts and the with the recommendation for opinions stakeholder of the target population. collaboration Moreover, this in the measurement is aligned withlearning of socioemotional the recommendation fordevelopmen- to be consistent, stakeholder collaboration tally appropriate and culturally sensitive [38]. Furthermore, we evaluated the develop- in the measurement of socioemotional learning to be consistent, response mentally processesappropriate and consequencesand culturally sensitive the of administering [38].test Furthermore, to verify thatwetheevaluated scores were the not re- sponse biased processes in any way. and consequences Finally, this studyof administering suggests that ECAQ the testcould to verify be athat the scores quicker were and easier not biased in any questionnaire way. Finally, to administer, this study to in comparison suggests others that are ECAQ could widely beina research. used quicker and For easier questionnaire to administer, in comparison to others that are widely used in re- search. For instance, administering the Affect Knowledge Test (AKT) [39,40]) or the Emo- tion Matching Task (EMT) [41,42] requires 15–20 min per child, whereas the average time for the ECAQ is 7 min. Additionally, for the AKT or the EMT, the researcher administers the test to the children; on the contrary, for the ECAQ, the teachers are the respondents. Educ. Sci. 2022, 12, 489 14 of 19 instance, administering the Affect Knowledge Test (AKT) [39,40]) or the Emotion Matching Task (EMT) [41,42] requires 15–20 min per child, whereas the average time for the ECAQ is 7 min. Additionally, for the AKT or the EMT, the researcher administers the test to the children; on the contrary, for the ECAQ, the teachers are the respondents. This has several advantages: the criteria are the same for all the children in the same class, the teachers know them better than an outside researcher, and the method is more economical. Our study of the internal structure of the ECAQ yielded positive evidence for only one general factor of competence. Although, a priori, our approach started from the development of a questionnaire based on the five competences proposed by Bisquerra [22], this result could be considered positive. The questionnaire contemplates all the domains of the necessary construct, and therefore, finding a unidimensional structure could be considered a success. We should bear in mind that the author himself indicates that the five competences included in their model overlap in some way with each other, forming as a whole the construct of emotional competence. It should be noted that discerning between emotional competences remains challenging at such early ages. In fact, the concepts of emotional intelligence and emotional competence and the theory surrounding both concepts have been evolving over the last three decades, generating extensive knowledge that allows for the improvement of emotional education [43]. However, both constructs are constantly under debate [23]. This implies that the different instruments that are emerging to measure them are likely to differ, even slightly, in what they are measuring and that it is difficult to know whether we are measuring emotional competence partially or completely. Different scales are used to measure emotional competence, often developed ad hoc to measure the outcome of a program or an intervention [44–46]. This makes it difficult to compare the results of such interventions as well as the benefits of one or another instrument. In spite of this, the predictive value of measuring social competence in the preschool stage is relevant [47] and it is therefore necessary to continue advancing in this field despite the complexity of comparison between instruments and constructs [48] and to try to arrive at a consensual conceptual framework [49]. As has been said, the ECAQ is a simple, short and easy-to-answer questionnaire comprising 30 questions. According to our study, the questionnaire is capable of yielding valid and reliable scores. Therefore, the ECAQ could be a good candidate for large-scale studies and health surveys to map where children’s emotional competences are the weakest. This would allow us to prioritize where to intervene to boost those competences related to health, especially mental health [50,51], in a way that contributes to reducing social inequalities in health [52,53]. In addition, the high-reliability values obtained would even allow for taking individual decisions following the criterion of Nunnally [37]. Regarding validity, as expected, the total score of the ECAQ correlated positively with the Emotional Regulation subscale of the ERC and the total scores of the SRS and SCS-T. The observed negative correlation with the Lability and Negativity subscale of the ERC was also in line with expectations. This is because emotional lability and negativity involve the difficulty to regulate oneself emotionally and to enhance positive moods, and emotional competence includes the ability to regulate unpleasant emotions and to enhance pleasant ones [22]. The ECAQ focuses exclusively on emotional competence, whereas the SRS incorporates items related to health promotion, and the SCS-T includes items related to academic skills. Given that the aim is to assess emotional competence, the fact that aspects other than emotional competence are not assessed is a positive aspect. In this way, the results also respond to the construct to be measured without being contaminated by other aspects that, although related to emotional competence [3], are not part of it. At the same time, the ECAQ incorporates several dimensions of emotional competence, which differentiates it from questionnaires such as the ERC, which focuses solely on emotional regulation, leaving aside key aspects of emotional competence (e.g., emotional regulation). This study has some limitations: (1) When selecting the items in the development process, we did not review any questionnaire that required payment to access it. Notwith- standing, more than 150 tests, scales and questionnaires were reviewed and experts in Educ. Sci. 2022, 12, 489 15 of 19 emotional education and teachers were asked if there was any key aspect missing. (2) We gathered the opinions of just four teachers for the evidence of validity based on test content and of just three for the evidence based on response processes. However, most items were adapted from other validated tools, and we assured the participation of a teacher from each age group (3- to 5-year-old children). In addition, from the 56 items that these teachers val- ued, 47 items (84%) were clear to all of them and so self-explanatory. (3) This questionnaire would not be appropriate for children with special educational needs; however, we will study it further to see how it could be adapted for them. (4) We focused on studying the consequences of administering the questionnaire, but the consequences of testing should also include the effect of communicating the results and making decisions based on them. (5) We only have data on children in the city of Barcelona. This implies that our results may not be representative of other realities. (6) For the study of the relations to other variables, we have a relatively small sample if we compare it with the sample used to study the internal structure of the ECAQ. Despite this, we have enough power to perform the aforementioned analysis. (7) The fact that families have not been able to participate could be a limitation for not having a complete vision of the child. However, some studies suggest that early childhood teachers, with appropriate training and support, can make reliable judgments about children’s competence [54]. (8) Related to the limitations of the questionnaire itself, it must be said that it was designed for epidemiological studies and therefore the time commitment in such a case may be high overall if teachers have to answer the questionnaire for 20–25 students in their class. In conclusion, given the importance of emotional competence in mental health [1] and the need to know its distribution over the population, and taking into account the creation of school interventions to increase it [3] and the need to evaluate whether they are effective [8] in improving emotional competence, the ECAQ is postulated as a good instrument. This instrument, the development and validation of which is presented in this study, covers the unmet public health need for a quick questionnaire answered by teachers to assess the five emotional competences proposed by Bisquerra [22] in children aged 3 to 5 years. Obtaining evidence of validity based on the test content, response processes and consequences was essential to develop the ECAQ. As has been seen, the ECAQ has shown favorable evidence of psychometric quality. Therefore, the ECAQ could be considered a psychometrically sound questionnaire to evaluate the emotional competence of three- to five-year-old children based on their teachers’ records. From a public health perspective, having a quick and easy-to-apply instrument that is validated with good psychometric properties in a big sample could allow future research to evaluate the population of 3- to 5-year-olds in cities of Spain. As the ECAQ can be applied on a large scale, this could allow researchers and policymakers to identify which areas of a city require more effort to increase the emotional competence of the children. This questionnaire can also be used to evaluate the interventions and programs implemented at these ages, which is essential in public health [8], allowing us to know if they improve the emotional competence of the target population as well as to understand the challenges and limitations of implementation efforts in order to sustain and improve them [55]. Supplementary Materials: The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/educsci12070489/s1, Figure S1. Frequencies of ECAQ; Table S1. Descriptive Statistics for ECAQ; outputs from analysis; relation with other variables; test-retest. Author Contributions: Conceptualization, M.B., A.A.-B., M.B.-P. and A.E.; methodology, M.B., A.A.-B. and A.E.; validation, M.B., A.A.-B., M.B.-P., C.C. and A.E.; formal analysis, A.A.-B., M.B. and A.E.; writing—original draft, M.B., A.A.-B. and A.E.; writing—review and editing, M.B., A.A.-B., M.B.-P., C.C. and A.E.; supervision, A.E. All authors have read and agreed to the published version of the manuscript. Funding: This research received no external funding. Educ. Sci. 2022, 12, 489 16 of 19 Institutional Review Board Statement: The study was undertaken in accordance with the Helsinki declaration for research involving human subjects and was approved by the Ethics and Research Committee of Parc Salut Mar under the number 2019/8508/I. Informed Consent Statement: As the ethics committee accepted, informed consent was unnecessary from the children’s families since data were anonymized and children could not be connected to questionnaire answers. The teachers, after requesting and obtaining permission from the schools in which they work to participate in the research study, signed an informed consent form indicating that they were voluntarily participating in the study and had received adequate and sufficient information regarding the processing of their personal data. Data Availability Statement: Data available upon request: Montse Bartroli (mbartrol@aspb.cat). Acknowledgments: We acknowledge Carme Viladrich, Jon Berastegui, Mireia Cabero, Gemma Filella, Raquel Gilar, Alicia Puente, Olga Juárez and all the teachers who have collaborated in the creation of this questionnaire. This paper forms part of the doctoral dissertation of Montse Bartroli at the Universitat Pompeu Fabra. Conflicts of Interest: The authors declare no conflict of interest. References 1. Allen, J.; Balfour, R.; Bell, R.; Marmot, M. Social determinants of mental health. Int. Rev. Psychiatry 2014, 26, 392–407. [CrossRef] [PubMed] 2. Shi, W.; Yuan, G.F.; Hall, B.J.; Liu, X.; Su, Y.; Zhao, L.; Jia, P. Mental Disorders and Emotional Competence Among Chinese Adolescents Before and during COVID-19 Pandemic: A Longitudinal Mediation Model. Front. Public Health 2021, 9, 767004. [CrossRef] [PubMed] 3. Durlak, J.A.; Weissberg, R.P.; Dymnicki, A.B.; Taylor, R.D.; Schellinger, K.B. The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Interventions. Child Dev. 2011, 82, 405–432. [CrossRef] [PubMed] 4. Llorent, V.J.; González-Gómez, A.L.; Farrington, D.P.; Zych, I. Social and emotional competencies and empathy as predictors of literacy competence. Psicothema 2020, 32, 47–53. [CrossRef] [PubMed] 5. Catalano, R.F.; Berglund, M.L.; Ryan, J.A.M.; Lonczak, H.S.; Hawkins, J.D. Positive Youth Development in the United States: Research Findings on Evaluations of Positive Youth Development Programs. Ann. Am. Acad. Pol. Soc. Sci. 2004, 591, 98–124. [CrossRef] 6. Taylor, R.D.; Oberle, E.; Durlak, J.A.; Weissberg, R.P. Promoting Positive Youth Development through School-Based Social and Emotional Learning Interventions: A Meta-Analysis of Follow-Up Effects. Child Dev. 2017, 88, 1156–1171. [CrossRef] 7. Ahmad, S.; Waldie, K.E.; Morton, S.M.B.; Peterson, E.R. Do Patterns of Levels of Socio-emotional Competence during Early Childhood Predict Executive Function at 4.5 Years? Child Psychiatry Hum. Dev. 2022, 53, 448–457. [CrossRef] 8. López, M.J.; Olmo, M.M.D.; Pérez-Giménez, A.; Nebot, M. Diseños evaluativos en salud pública: Aspectos metodológicos. Gac. Sanit. 2011, 25, 9–16. [CrossRef] 9. Halle, T.G.; Darling-Churchill, K.E. Darling-Churchill. Review of measures of social and emotional development. J. Appl. Dev. Psychol. 2016, 45, 8–18. [CrossRef] 10. Roben, C.K.P.; Cole, P.M.; Armstrong, L.M. Longitudinal Relations Among Language Skills, Anger Expression, and Regulatory Strategies in Early Childhood. Child Dev. 2013, 84, 891–905. [CrossRef] 11. Nix, R.L.; Bierman, K.L.; Domitrovich, C.E.; Gill, S. Promoting Children’s Social-Emotional Skills in Preschool Can Enhance Aca-demic and Behavioral Functioning in Kindergarten: Findings from Head Start REDI. Early Educ. Dev. 2013, 24, 1000–1019. [CrossRef] 12. Finlon, K.J.; Izard, C.E.; Seidenfeld, A.; Johnson, S.R.; Cavadel, E.W.; Ewing, E.S.K.; Morgan, J.K. Emotion-based preventive intervention: Effec-tively promoting emotion knowledge and adaptive behavior among at-risk preschoolers. Dev. Psychopathol. 2015, 27, 1353–1365. [CrossRef] [PubMed] 13. Waters, S.F.; Virmani, E.A.; Thompson, R.A.; Meyer, S.; Raikes, H.A.; Jochem, R. Emotion Regulation and Attachment: Unpacking Two Constructs and Their Association. J. Psychopathol. Behav. Assess. 2010, 32, 37–47. [CrossRef] [PubMed] 14. Torres, M.M.; Domitrovich, C.E.; Bierman, K.L. Preschool interpersonal relationships predict kindergarten achievement: Mediat- ed by gains in emotion knowledge. J. Appl. Dev. Psychol. 2015, 39, 44–52. [CrossRef] [PubMed] 15. Weiland, C.; Yoshikawa, H. Impacts of a Prekindergarten Program on Children’s Mathematics, Language, Literacy, Executive- Function, and Emotional Skills. Child Dev. 2013, 84, 2112–2130. [CrossRef] [PubMed] 16. Williford, A.P.; Vick Whittaker, J.E.; Vitiello, V.E.; Downer, J.T. Children’s Engagement Within the Preschool Classroom and Their Development of Self-Regulation. Early Educ. Dev. 2013, 24, 162–187. [CrossRef] 17. Chinekesh, A.; Kamalian, M.; Eltemasi, M.; Chinekesh, S.; Alavi, M. The Effect of Group Play Therapy on Social-Emotional Skills in Pre-School Children. Glob. J. Health Sci. 2013, 6, 163–167. [CrossRef] 18. Spritz, B.L.; Sandberg, E.H.; Maher, E.; Zajdel, R.T. Models of Emotion Skills and Social Competence in the Head Start Class-room. Early Educ. Dev. 2010, 21, 495–516. [CrossRef] Educ. Sci. 2022, 12, 489 17 of 19 19. Onchwari, G.; Keengwe, J. Examining the Relationship of Children’s Behavior to Emotion Regulation Ability. Early Child Educ. J. 2011, 39, 279–284. [CrossRef] 20. Miller, A.R.; Johnston, C.; Klassen, A.F.; Fine, S.; Papsdorf, M. Family physicians’ involvement and self-reported comfort and skill in care of children with behavioral and emotional problems: A population-based survey. BMC Fam. Pract. 2005, 6, 12. [CrossRef] 21. Bernard, M.E.; Magnum, N.D.U. Wellbeing Survey (Teacher Form—Early Years), 2nd ed.; Australian Council for Educational Research: Melbourne, Australia, 2009. 22. Bisquerra, R. Educación emocional y competencias básicas para la vida. Rev. Investig. Educ. 2003, 21, 7–43. 23. Bisquerra, R.; Pérez, N. Las competencias emocionales. Educ. XXI 2007, 10, 61–82. [CrossRef] 24. AERA; APA; NCME. Standards for Educational and Psychological Testing; American Educational Research Association: Washington, DC, USA, 2014. 25. Schmitt, S.A.; Flay, B.R.; Lewis, K. A pilot evaluation of the Positive Action prekindergarten lessons. Early Child Dev. Care 2014, 184, 1978–1991. [CrossRef] 26. Corrigan, A. Social Competence Scale—Teacher Version‚ Grade 3/Year 4 (Fast Track Project Technical Report). 2003. Available online: http://www.fasttrackproject.org (accessed on 20 May 2022). 27. Shields, A.; Cicchetti, D. Emotion regulation among school-age children: The development and validation of a new criterion Q-sort scale. Dev. Psychol. 1997, 33, 906–916. [CrossRef] [PubMed] 28. Servera, M.; García-Delgar, B.; Cicchetti, D. The emotion regulation checklist: Spanish Balearic Islands translation. 2016, Unpublished manuscript. 29. Sireci, S.; Faulkner-Bond, M. Evidencia de validez basada en el contenido del test. Psicothema 2014, 26, 100–107. 30. Falzarano, M.; Pinto Zipp, G. Seeking consensus through the use of the Delphi technique in health sciences research. J. Allied Health 2013, 42, 99–105. 31. Bosque-prous, M.; Espelt, A.; Juárez, O.; Ramos, P.; Bartroli, M. Educación Emocional y Salud Pública. Psico 2015, 46, 409–411. [CrossRef] 32. Rosseel, Y. lavaan: An R Package for Structural Equation Modeling. J. Stat. Softw. 2012, 48, 1–36. [CrossRef] 33. Hu, L.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alterna-tives. Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. [CrossRef] 34. Marsh, H.W.; Hau, K.-T.; Wen, Z. In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler’s (1999) Findings. Struct. Equ. Model. A Multidiscip. J. 2004, 11, 320–341. [CrossRef] 35. Rodriguez, A.; Reise, S.P.; Haviland, M.G. Evaluating bifactor models: Calculating and interpreting statistical indices. Psychol. Methods 2016, 21, 137–150. [CrossRef] [PubMed] 36. Viladrich, C.; Angulo-brunet, A.; Doval, E. A journey around alpha and omega to estimate internal consistency reliability. An. Psicol. 2017, 33, 755–782. [CrossRef] 37. Nunnally, J.C. Psychometric Theory; McGrawHill: New York, NY, USA, 1978. 38. Prado-Galbarro, F.J.; Pérez-Ferrer, C.; Ortigoza, A.; López-Olmedo, N.P.; Braverman-Bronstein, A.; Rojas-Martínez, R.; de Castro, F.; Barrientos-Gutiérrez, T. Early childhood development and urban environment in Mexico. PLoS ONE 2021, 16, e0259946. [CrossRef] [PubMed] 39. Locke, R.L.; Lang, N.J. Emotion knowledge and attentional differences in preschoolers showing context-inappropriate anger. Percept. Mot. Ski. 2016, 123, 46–63. [CrossRef] 40. Sette, S.; Baumgartner, E.; Laghi, F.; Coplan, R.J. The role of emotion knowledge in the links between shyness and children’s socio-emotional functioning at preschool. Br. J. Dev. Psychol. 2016, 34, 471–488. [CrossRef] 41. Di Maggio, R.; Zappulla, C.; Pace, U. The Relationship Between Emotion Knowledge, Emotion Regulation and Adjustment in Preschoolers: A Mediation Model. J. Child Fam. Stud. 2016, 25, 2626–2635. [CrossRef] 42. Alonso-Alberca, N.; Vergara, A.I.; Fernández-Berrocal, P.; Johnson, S.R.; Izard, C.E. The adaptation and validation of the Emotion Matching Task for preschool children in Spain. Int. J. Behav. Dev. 2012, 36, 489–494. [CrossRef] 43. Mayer, J.D.; Caruso, D.R.; Salovey, P. The Ability Model of Emotional Intelligence: Principles and Updates. Emot. Rev. 2016, 8, 290–300. [CrossRef] 44. Sancassiani, F.; Pintus, E.; Holte, A.; Paulus, P.; Moro, M.F.; Cossu, G.; Angermeyer, M.C.; Carta, M.G.; Lindert, J. Enhancing the Emotional and Social Skills of the Youth to Promote their Wellbeing and Positive Development: A Systematic Review of Universal School-based Randomized Controlled Trials. Clin. Pract. Epidemiol. Ment. Health 2015, 11, 21–40. [CrossRef] 45. O’Reilly, M.; Svirydzenka, N.; Adams, S.; Dogra, N. Review of mental health promotion interventions in schools. Soc. Psychiatry Psychiatr. Epidemiol. 2018, 53, 647–662. [CrossRef] 46. Anziom, B.; Strader, S.; Sanou, A.S.; Chew, P. Without Assumptions: Development of a Socio-Emotional Learning Framework That Reflects Community Values in Cameroon. Front. Public Health 2021, 9, 602546. [CrossRef] [PubMed] 47. Jones, D.E.; Greenberg, M.; Crowley, M. Early Social-Emotional Functioning and Public Health: The Relationship Between Kindergarten Social Competence and Future Wellness. Am. J. Public Health 2015, 105, 2283–2290. [CrossRef] [PubMed] 48. Humphrey, N.; Kalambouka, A.; Wigelsworth, M.; Lendrum, A.; Deighton, J.; Wolpert, M. Measures of Social and Emotional Skills for Children and Young People: A Systematic Review. Educ. Psychol. Meas. 2011, 71, 617–637. [CrossRef] Educ. Sci. 2022, 12, 489 18 of 19 49. Jones, S.; Zaslow, M.; Darling-Churchill, K.E.; Halle, T.G. Assessing early childhood social and emotional development: Key conceptual and measurement issues. J. Appl. Dev. Psychol. 2016, 45, 42–48. [CrossRef] 50. Neil, A.L.; Christensen, H. Efficacy and effectiveness of school-based prevention and early intervention programs for anxiety. Clin. Psychol. Rev. 2009, 29, 208–215. [CrossRef] [PubMed] 51. Fernández-Martínez, I.; Morales, A.; Espada, J.P.; Essau, C.A.; Orgilés, M. Effectiveness of the program super skills for life in reducing symptoms of anxiety and depression in young spanish children. Psicothema 2019, 31, 298–304. [CrossRef] [PubMed] 52. Meilstrup, C.; Holstein, B.E.; Nielsen, L.; Due, P.; Koushede, V. Self-efficacy and social competence reduce socioeconomic inequality in emotional symptoms among schoolchildren. Eur. J. Public Health 2020, 30, 80–85. [CrossRef] 53. Gedikoglu, M. Social and Emotional Learning. An Evidence Review and Synthesis of Key Issues; Education Policy Institute: London, UK, 2021. 54. Darling-Churchill, K.E.; Lippman, L. Early childhood social and emotional development: Advancing the field of measurement. J. Appl. Dev. Psychol. 2016, 45, 1–7. [CrossRef] 55. Mahoney, J.L.; Weissberg, R.P.; Greenberg, M.T.; Dusenbury, L.; Jagers, R.J.; Niemi, K.; Schlinger, M.; Schlund, J.; Shriver, T.P.; Van Ausdal, K.; et al. Systemic social and emotional learning: Promoting educational success for all preschool to high school students. Am. Psychol. 2021, 76, 1128–1142. [CrossRef]