RESEARCH ARTICLE Warning signs of preschool victimization using the strengths and difficulties questionnaire: Prevalence and individual and family risk factors Jose-Blas Navarro ID1*, Meritxell Fernández2, Núria de la Osa2, Eva Penelo1, Lourdes Ezpeleta2 a1111111111 1 Department of Psychobiology and Methodology of the Health Sciences, Universitat Autònoma de a1111111111 Barcelona, Bellaterra, Spain, 2 Unit of Epidemiology and Diagnosis in Developmental Psychopathology, a1111111111 Department of Clinical and Health Psychology, Universitat Autònoma de Barcelona, Bellaterra, Spain a1111111111 * joseblas.navarro@uab.cat a1111111111 Abstract OPEN ACCESS Citation: Navarro J-B, Fernández M, de la Osa N, Introduction Penelo E, Ezpeleta L (2019) Warning signs of School victimization by peers is an important social problem with serious short- and long- preschool victimization using the strengths and term consequences poorly studied at preschool ages, which can lead to school bullying with- difficulties questionnaire: Prevalence and individual and family risk factors. PLoS ONE 14(8): out timely intervention. Longitudinal data was used to determine the prevalence of warning e0221580. https://doi.org/10.1371/journal. signs of preschool peer victimization and its individual and family risk factors. pone.0221580 Editor: Kenji Hashimoto, Chiba Daigaku, JAPAN Methods Received: April 21, 2019 Data was obtained from 577 community preschoolers. School victimization was measured Accepted: August 11, 2019 using the Strengths and Difficulties Questionnaire (SDQ) administered to parents and teach- ers of children at ages 4 and 5. Risk factors for the child (demographics, conduct and emo- Published: August 23, 2019 tional problems, aggressiveness) and the family (maternal problems during pregnancy and Copyright: © 2019 Navarro et al. This is an open early development, parenting styles, adaptive functioning and parents’ problems) were pre- access article distributed under the terms of the Creative Commons Attribution License, which viously recorded at 3 years old. permits unrestricted use, distribution, and reproduction in any medium, provided the original Results author and source are credited. Combined information from parents and teachers showed that 4.2% of preschoolers pre- Data Availability Statement: Data cannot be made publicly available due to ethical restrictions sented warning signs of victimization at ages 4 and 5. Low socioeconomic status, poor emo- protecting the confidentiality of the families tional control, early problems making friends and low level of parenting education in social involved (Ethics Commission of Animal and norms at age 3 predicted later victimization at ages 4 and 5 (AUC = .78). Human Experimentation of the Universitat Autònoma de Barcelona). Data are available to interested researchers after signing a consent of Conclusion confidentiality form as the authors had previously Peer victimization affects a considerable percentage of preschoolers. Early detection may signed to obtain the data from the families of the sample. Researchers must be working in clinical help to reduce the risk of escalation. child psychology in a public funded project. To request the anonymous data, please contact the PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 1 / 19 Warning signs of preschool victimization Department of Clinical and Health Psychology Introduction of the Universitat Autònoma de Barcelona (d.psic.salut@uab.cat). Peer victimization is a developmentally salient interpersonal stressor that refers to the experi- ence of being the recipient of peer intimidation and harassment or being the target of physical, Funding: This work was supported by the Spanish Ministry of Economy and Competitiveness social, emotional or psychological harm from a peer [1]. As a form of aggressive behavior there (MINECO/FEDER) [grant PSI2012-32695] and by is the risk of escalation and if the victimization is repeated and intentional and there exists an the Spanish Ministry of Science, Innovation and imbalance of power between aggressors and victim, it takes the form of bullying [2]. School Universities (MICINN/FEDER) [grant PGC2018- victimization is an important social problem with serious short-term consequences for victims’ 095239-B-I00]. physical and psychological health and negative long-term effects on their future psychosocial Competing interests: The authors have declared adjustment as adults. that no competing interests exist. The cumulative consequences of early victimization may emerge at later points in children’s school careers and have been well documented [3,4]. In cases where the victimization becomes chronic, the consequences may have an even greater effect on children’s achievements and learning [5]. Peer victimization has been associated with emotional and behavioral problems over time that are beyond the child’s pre-existing difficulties [6], with children who are victim- ized at risk of a wide range of adjustment difficulties that include: self-harm [7], loneliness, social avoidance, self-blame [8], suicide ideation [9], anxiety and depression [10], social maladjustment, loneliness [11], lack of close peer relationships [12], low self-esteem [13], poor physical health, psychosomatic complaints, behavioral dificulties and deficient emotional adjustment [13,14]. Other important findings show there is also a potential significative impact on adult adjustment outcomes [15] such as problems with doing housework and managing money [16], trouble forming new social relationships, less social support, poorer family func- tioning and lower levels of education [17]. Early diagnosis and timely treatment of these problems at a young age will not only improve children’s adaptation at school and their emotional and social development [18], but will also help to prevent their impact in later years. Preschool and peer victimization Most of the research about peer victimization has focused on middle childhood or adolescence. Beyond the home, school is usually the main environment where children’s difficulties with social interactions with their peers can be primarily detected and subsequently assessed by adults and professionals [18]. During preschool and kindergarten, children learn how to build and maintain friendships, and they form opinions about who they like or dislike and they establish groups of consistent play partners [19]. Experts on child development strongly agree on the importance of acquiring cognitive and social-emotional competencies at the pre-school stage [20]. The manifestation of peer victimization in the preschool years is similar to that of school- age children in many ways but differs in others depending on the child’s development of social cognition [21]. Younger students usually tend to resort to physical aggression, possibly because they have not yet developed the required sophistication of verbal or social skills to obtain what they want. With maturation, children start utilizing verbal skills to engage in more subtle forms of verbal aggression [22,23] and/or develop more complex social skills such as assessing and manipulating social situations, and they can also engage in indirect forms of aggression [22]. For these reasons it is important to study peer victimization from preschool ages. Prevalence and measure Although the most severe forms of peer victimization seem to have reduced in many coun- tries over the last few years [24], efforts to detect and prevent this type of aggressive behavior PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 2 / 19 Warning signs of preschool victimization in schools are still essential. There is an abundance of large-scale studies on elementary and high school children [12,25], but less studies have been carried out on preschoolers. The literature shows that even at these early ages peer victimization occurs frequently and is asso- ciated with poor psychosocial well-being [26]. Ilola et al. [27] found a 4% six-month preva- lence of frequently bullied children as reported by the parents of a sample of 931 4-year-olds who attended pediatric checkups. In another large population-based study of 5- to 6-year- olds that used reports from parents and teachers, a similar 4% of the children were victims [10]. A problem like peer victimization, which remains hidden in most cases, requires informa- tion from as many sources as possible. The Strengths and Difficulties Questionnaire (SDQ) [28] is a tool with good psychometric properties that is widely used to screen for psychiatric disorders and its version for children aged 3 to 4 years old has been utilized at preschool age [29,30]. The SDQ has versions for parents and teachers to account for discrepancies between informants, since different informants observe children’s behavior in different settings [31]. Parents and teachers are the ones who spend most time with pre-school aged children. Given the preventive aim of early detection, both reports are indispensable under the Attribution Bias Context Model [32], which posits that informant discrepancies are indicative of cross- contextual variability in children’s behavior and informants’ perspectives on this behavior. The SDQ includes a peer problems scale, which contains a specific item for peer victimization. If this questionnaire is widely used and registers the presence of peer victimization, it could be used for identifying early warning signs of problematic behaviors indicating that the child is a target of aggressive victimization behaviors, and for studying their risk factors. A warning sign is an observable manifestation that hints of the existence of something that may impact nega- tively on the person, in this case peer victimization. Risk factors To help designing preventive and intervention strategies against peer victimization, it is crucial to determine the risk factors that enable the prediction of the onset of victimization to facilitate an early identification of infants at risk of becoming victims [33]. Several longitudinal studies have identified factors that relate children’s personal characteristics and their home environ- ment with an increased risk of being victimized [34]. The social-ecological model posits that peer victimization involvement is determined by the multiple systems in which youth are embedded [35]. First introduced in the late 1970s [36], this ecological model was initially applied to child abuse [37] and subsequently to youth vio- lence [38]. The model explores the relationship between individual and contextual factors and considers violence as the product of multiple levels of influence on behavior [39]. Understand- ing how these factors are related to peer victimization is one of the essential steps in the public health approach to preventing this violence [40]. The first level of the ecological model focuses on the intrinsic characteristics of the individ- ual that might increase the likelihood of their either becoming a victim or a perpetrator of vio- lence, and the guiding assumption for these factors is that victimized children behave in ways that invite or reinforce attacks against them [41]. It has been suggested that aggressors “often have a positive outlook on the use of violence to solve problematic situations or get what they want” [42]. This follows the theoretical assumptions underlying proactive aggression. Bullies can be quite adept at identifying victims who will either not retaliate or will be ineffective in their efforts to retaliate, making them more desirable targets [43]. Some children may have temperamental, behavioral and cognitive characteristics that evoke aggressive behavior from others and make them more prone to victimization. PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 3 / 19 Warning signs of preschool victimization Among children and adolescents, longer exposure to peer victimization has been associated with higher emotional sensitivity, anger, fear and internalizing and unstable affective tempera- ments [44]. Similarly, psychopathology may contribute to victimization. Over time internaliz- ing (withdrawal, anxiety/depression and hovering peer entry style) and externalizing (aggression, argumentativeness, dishonesty, pushy peer entry style and disruptiveness) behav- ior problems are associated with victimization [45]. However, prospective studies of young children that examine such antecedent effects are largely lacking [46]. Few studies have tackled the cognitive characteristics of the victimized children. In a Brazilian sample was found that 10- to 11-year-old child peer victims presented difficulties in executive functioning and showed lower cognitive flexibility [47]. Previous studies regarding victimization and ethnic and/or racial differences have produced mixed or inconclusive results [48]. Some studies report that minority youth are more likely to be victimized [49], whereas others have found that the percentage of immigrants and the racial variability of school children do not seem to be risk factors for being victimized [50]. The second level of the ecological model is related to the family environment and explores how proximal social relationships, for example relations with family members, increase the risk of victimization [40]. The impact and relation of family characteristics as risk factors of victimization has gained increased attention in recent years. It has been shown that parental characteristics and adverse lived experiences may have an impact on children, facilitating peer victimization [51]. Several studies have shown that peer victimization is associated with inconsistent, punitive, hostile and/or abusive parenting, high negative expressiveness and high levels of family conflict or violence [52,53]. Also, children growing up in low or mean socioeconomic status (SES) families are at increased risk of victimization [54]. Furthermore, victims from schools in low-SES communities or with a large proportion of students from low-SES families suffer worse long-term mental illness as consequence of the victimization compared to victims from richer families [54]. Additionally, severe prenatal family adversity significantly increased the risk of peer victimization at school [55]. Also, children whose relatives were abused during childhood have an increased risk of being bullied, which may indicate an intergenerational transmission of maltreatment [56]. Last, parental psychopa- thology also affects the child’s risk of being victimized through complex pathways, such as genetic, neurobiological (e.g., dysregulated stress systems) and social (e.g., modelling) ones [57]. Purpose of the study The present investigation is based on the premise that children’s behavioral attributes and fam- ily factors contribute to their victimization, and has the aim of identifying the children with an increased risk of becoming a victim at the youngest possible age to facilitate timely prevention of the problem. Awareness should be raised about how the instruments at use allow us to iden- tify the problem and the characteristics of the identified children to determine their usefulness in screening for peer victimization [58]. Using a prospective longitudinal design, the present study seeks to contribute new knowl- edge about the early individual and family risk factors of being peer victimized during the pre- school period and seeks to obtain an estimate of the prevalence of these warning signs of peer victimization at 4 to 5 years old, using a widely used screening instrument (SDQ). In align- ment with Bronfenbrenner’s ecological model [36], 35 individual and family factors were ini- tially selected to identify which were the most predictive of peer victimization. Selection of the final set of risk factors was made through a predictive modelling approach. PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 4 / 19 Warning signs of preschool victimization Materials and methods Study design and procedure The data was part of a large-scale longitudinal study of behaviour problems in preschool chil- dren from age 3, who were screened for behaviour problems and followed up annually until age 9 (the design procedure is detailed in [59]). The two-phase design used involved selecting first a random sample of 2,283 children from the census of in-school 3-year-old preschoolers in Barce- lona (Spain) during the 2009–10 academic year (N = 13,578 children). The sample size was determined so that the study was able to detect associations between exposure to different risk factors and the presence of psychopathology equal to or greater than OR = 2.0, accepting an alpha risk of 5% and a statistical power of 80%. Epidemiological studies indicate that the preva- lence of psychopathology in children and adolescents (range of ages from 6 to 16 years) of the general population is approximately 15% (p = 0.15) [60]. Likewise, a multiple correlation between covariates of 0.4 (R2 = 0.16) is assumed. Some corrections were applied to the obtained sample: a) correction for homogeneity of children within the same classroom (random clustered sampling) increasing the sample by 20%; b) Correction because double stage design. It was established that screening would classify children into two equally sized groups (50%). Because only 30% of the group of children with negative screening will be followed, it is necessary to pro- portionally increase the initial number of children to be recruited; and c) Correction for attrition due to refusal of the school to continue participating (assuming 20% of centers will abandon) and due to refusal of subjects, assuming 50% of families will leave the study (as indicated by sim- ilar follow-up studies carried out in advanced countries). Exclusion criteria included the child having a generalized developmental disorder, the parents having reading problems or not understanding neither Spanish nor Catalan languages and having planned a move outside Bar- celona. All the parents of the initially selected pupils were invited to participate in the study, from which 1,341 families (58.7%) agreed to participate. There were no sex or socioeconomic differences (p = .95 and p = .182) between those who agreed to participate and those who declined, although more participants than refusals attended private schools (p < .001). The second phase of the sampling consisted in a screening of behavioral problems carried out using the SDQ conduct problems scale [28] (see below) plus four symptoms of DSM-IV oppositional defiant disorder (ODD) (the principal disorder of interest in the original study) not present in the questionnaire (deliberately annoys people, blames others, touchy, angry and resentful). To ensure the presence of children with behavior problems in this second phase, all of those who screened positive for behavioral problems and a random sample of around 30% of children who screened negative (the number of children needed in the negative screening score was calculated to guarantee statistical power) were invited to continue. The final second phase sample included 622 families (10.6% of those invited declined to participate). No differences were found on comparing participants and refusals either by sex (p = .820) or school type (p = .850). The families were interviewed at the school. Interviewers were previously trained and were blind to the children’s screening group. The teachers were asked to complete the question- naires before the end of the academic year. The Fig 1 shows the sampling procedure. The research project was approved by the Ethics Commission on Animal and Human Experimentation from the Autonomous University of Barcelona. The parents of the partici- pants children were informed and gave written consent to participate in the study. Instrumentation Outcome measures at ages 4 and 5. Strengths and Difficulties Questionnaire for ages 3–4 (SDQ3-4) [28] for parents and teachers. It is a screening questionnaire for behavioural and PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 5 / 19 Warning signs of preschool victimization Fig 1. Sampling procedure of the study. https://doi.org/10.1371/journal.pone.0221580.g001 emotional problems applied for the purposes of this study when the children were 3 (parents version for control variables), 4 and 5 years old (parents and teachers for the outcome vari- able). Item 19 - ‘Picked on or bullied by other children’, coded as 0: not true; 1: somewhat true; 2: certainly true—answered by parents and teachers was selected as outcome. We conceptual- ized that if parents or teachers scored 1 or 2 on this item at both follow-ups at ages 4 and 5 this PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 6 / 19 Warning signs of preschool victimization would be a warning sign for investigating the child’s possible victimization; that is, if the prob- lem as reported by any of the informants was sustained over time, the case would be consid- ered as victimized. Good psychometric properties in the sample have been reported [61], with Cronbach’s alpha = .73 for parents and .82 for teachers at age 3, and slightly better values for ages 4 and 5. Predictive measures at age 3. Child Behavior Checklist (CBCL/1½-5) [62]. It is a ques- tionnaire of behavioral and emotional problems reported by parents through 100 items with 3 response options (0: not true, 1: somewhat/sometimes true, 2: very true/often true). Internaliz- ing and externalizing empirical scales, plus the DSM-oriented scales depressive, anxiety, devel- opmental, attention deficit/hyperactivity and oppositional defiant problems, were used for the analyses. Higher scores indicate higher symptomatology. Behavior Rating Inventory of Executive Function—Preschool Version (BRIEF-P) [63]. It is a standardized questionnaire that assesses behaviors reflecting executive functions in daily life in preschool children. The instrument consists of 63 items on a 3-point scale (1: Never, 2: Sometimes, and 3: Often). Two dimensions which may be associated with social problems, Inhibit (I) and Emotional Control (EM), were used and higher scores indicated greater diffi- culties. Teachers answered the questionnaire when the children were aged 3. Good psychomet- ric properties have been reported [64]. Children’s Aggression Scale (CAS) [65]. It is a questionnaire that assesses aggressive behav- ior with 22 items on a 5-point Likert-type scale (0: never to 4: many days). The total score of the teachers’ responses was used as a global index of aggressive behavior. Higher scores indi- cate greater aggressive behavior. Children’s Behaviour Questionnaire for ages 3–7 years (CBQ3-7) [66]. It is a questionnaire that measures reactive and self-regulative temperament, with 94 items answered by parents on a 7-point Likert-type scale, ranging from 1 (extremely untrue) to 7 (extremely true). The 3 broad dimensions of temperament: negative affectivity, effortful control and surgency were considered. Higher scores indicate more marked traits. The reported psychometric properties are good [67]. Schedule for Risk Factors (SRF) [68]. It is a computerized structured interview conceived as a compendium of the potential areas of risk of psychopathology that should be evaluated in children. Demographic information including SES [69], pregnancy events, early development and history of abuse suffered by family members were used for this analysis. These risks factors were registered as present/absent as reported by parents. The reported psychometric proper- ties are good [70]. Alabama Parenting Questionnaire for Preschool (APQ-Pr) [71]. It was used to assess par- enting practices. It contains 24-item (1: Never to 5: Always) which measures three dimensions of parenting: positive discipline techniques, inconsistent parenting and punitive parenting. Two additional scales extracted from the Evaluation des Pratiques Educatives Parentales–Pre- school and Primary School Form (EPEP–PPSF) [72], norms (6 items, labelled as Rules in the EPEP-PPSF) and autonomy (3 items), were added to the instrument with the same response format. In the original EPEP-PPSF, items assessing rules and autonomy emerged as two sepa- rated factors and jointly with items assessing positive parenting emerged as a second-order fac- tor of Support; the authors also found low but significant positive correlations between these three measures and desirable traits in the child’s personality. In our sample, an exploratory fac- tor analysis for categorical items conducted with MPlus8.1 showed that the 9-item and 2-factor model with Geomin rotation (χ2 [17] = 108.0, comparative fit index [CFI] = .990, root mean square error of approximation [RMSEA] = .091) for Norms and Autonomy items fitted better than a 9-item and 1-factor model (χ2 [9] = 721.7, CFI = .925, RMSEA = .212); scree test based on eigen values also supported the retention of two factors. All items had salient loading in the PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 7 / 19 Warning signs of preschool victimization Table 1. Descriptive statistics of individual predictors at age 3 and results of logistic regression models predicting warning signs of peer victimization at ages 4–5. Individual predictors Cronbach’s Warning signs Logistic regression α No Yes n (%) or Mean (SD) OR 95% CI OR Demographics (n = 577) Sex (male) 277 (50.1%) 12 (50.0%) 1.11 0.42; 2.93 Ethnicity (non-Caucasian) 58 (10.5%) 1 (4.2%) 0.18 0.02; 1.58 SES (low or medium-low) 110 (19.9%) 12 (50.0%) 5.15� 1.91; 13.94 CBCL 1½-5 (n = 573) Internalizing .83 7.2 (5.8) 10.5 (7.6) 1.08 1.01; 1.17 Externalizing .85 10.5 (6.1) 12.3 (7.4) 0.99 0.92; 1.07 Depressive problems .51 1.9 (2.0) 2.5 (1.9) 1.01 0.80; 1.27 Anxiety problems .65 3.0 (2.6) 4.3 (3.1) 1.05 0.82; 1.33 Developmental problems .65 2.7 (2.5) 4.5 (2.8) 1.20 1.01; 1.42 ADH problems .74 4.1 (2.6) 4.7 (2.8) 1.05 0.83; 1.33 Oppositional defiant problems .74 3.2 (2.2) 3.4 (2.5) 0.90 0.66; 1.24 BRIEF-P (n = 575) Inhibit .93 23.1 (7.0) 24.6 (7.8) 0.99 0.93; 1.05 Emotional control .88 12.1 (3.5) 14.0 (4.4) 1.14� 1.03; 1.26 CAS (n = 563) Total score .82 265.2 (20.5) 271.8 (23.7) 1.01 0.99; 1.03 CBQ (n = 574) Surgency .74 4.3 (0.8) 4.2 (0.7) 0.93 0.54; 1.60 Effortful control .79 5.2 (0.6) 5.5 (0.7) 2.01 0.87; 4.67 Negative affectivity .71 3.8 (0.7) 3.9 (0.7) 1.40 0.77; 2.56 Note: In brackets, for categorical predictors: risk category; in bold, results for predictors with p < .10 � p-value remained below .10 after False Discovery Rate ADHD: attention-deficit/hyperactivity disorder; SDQ: Strengths and Difficulties Questionnaire; CBCL: Child Behavior Checklist; BRIEF-P: Behavior Rating Inventory of Executive Functioning-Preschool version; CAS: Children’s Aggression Scale; CBQ: Children’s Behavior Questionnaire. https://doi.org/10.1371/journal.pone.0221580.t001 expected factor (λ � .65; p � .001). Moreover, in our sample a confirmatory factor analysis conducted with MPlus8.1 for the whole set of items, that is considering a 33-item and 5-factor model, showed acceptable fit (χ2 [485] = 1112.8, CFI = .945, RMSEA = .048), all factor loadings being at least moderate and statistically significant (λ � .34; p � .001); as expected, the higher values for factor correlations were found between norms, autonomy, and positive discipline techniques (r between .35 and .57). For all five measures used in the present study, higher scores indicate higher parental practices in the direction of the label of the scale. Adult Self-Report (ASR) [73]. It is a questionnaire of dimensional psychopathology for adults aged 18–59. It contains 126 items (0: not true, 1: somewhat or sometimes true, and 2: very true or often true) and was answered by the mothers. DSM-oriented scales were analyzed. Higher scores indicate higher symptomatology. Second column of Tables 1 and 2 show the Cronbach’s alpha values of the measures in the sample. Statistical analysis The software used for statistical analysis was Stata 15 for Windows. To reflect population prev- alence (as final participant selection was conditioned by the presence/absence of behavior PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 8 / 19 Warning signs of preschool victimization Table 2. Descriptive statistics of family and early predictors at age 3 and results of logistic regression models predicting warning signs of peer victimization at ages 4–5. Family and early predictors Cronbach’s Warning signs Logistic regression α No Yes N (%) or Mean (SD) OR 95% CI OR Risk Interview (n = 567) Pregnancy Maternal emotional problems (yes) 105 (19.3%) 10 (41.7%) 2.31 0.86; 6.21 Living with conflict (yes) 50 (9.2%) 3 (12.5%) 0.74 0.14; 3.76 Economic problems (yes) 16 (2.9%) 5 (20.8%) 5.11 1.33; 19.68 Early Development Postpartum depression (yes) 113 (20.8%) 9 (37.5%) 2.26 0.82; 6.17 Clumsy child (yes) 64 (11.6%) 6 (25.0%) 2.45 0.85; 7.07 Difficulty making friends (yes) 47 (8.5%) 5 (20.8%) 4.95� 1.47; 16.63 History of abuse Relative abused in childhood (yes) 54 (9.8%) 6 (25.0%) 6.87� 1.69; 27.89 Biological mother abused (yes) 36 (6.5%) 3 (12.5%) 0.24 0.04; 1.32 APQ-Pr (n = 561) Positive parenting .75 40.8 (4.2) 41.2 (3.5) 1.09 0.98; 1.22 Inconsistent parenting .62 6.9 (3.1) 6.8 (3.2) 0.94 0.80; 1.11 Punitive parenting .42 3.7 (1.9) 3.9 (2.3) 1.12 0.90; 1.40 Norms .88 22.4 (2.5) 20.1 (4.4) 0.74� 0.66; 0.83 Autonomy .81 9.5 (2.1) 9.4 (2.0) 1.11 0.88; 1.39 ASR-DSM Mother (n = 517) Depressive .77 3.2 (3.2) 5.3 (4.2) 1.05 0.88; 1.24 Anxiety .64 5.5 (2.4) 6.8 (2.6) 1.08 0.87; 1.35 Somatic .69 1.1 (1.7) 2.9 (3.9) 1.28 1.02; 1.62 Avoidant personality .66 2.2 (1.9) 3.4 (2.6) 1.18 0.91; 1.52 ADHD .75 3.9 (3.2) 4.8 (3.9) 0.86 0.67; 1.09 Antisocial personality .55 2.2 (2.1) 3.1 (2.5) 1.09 0.89; 1.33 Note: In brackets, for categorical predictors: risk category; in bold, results for predictors with p < .10 � p-value remained below .10 after False Discovery Rate. APQ-Pr: Alabama Parenting Questionnaire for Preschool; ASR-DSM: Adult Self-Report DSM scales; ADHD: attention-deficit/hyperactivity disorder https://doi.org/10.1371/journal.pone.0221580.t002 problems), the analyses were weighted by the inverse probability of being selected in the sec- ond phase of the sampling design. A two-stage modelling procedure [74] was used to determine the best predictive model of 4- to 5-year-old warning signs of peer victimization from the 35 predictors pre-selected at age 3 according to the literature. First, several independent binary logistic regression models were estimated, grouping the predictors by content (demographics, individual and family risk fac- tors). Because of the large number of predictors included in this first stage, the false discovery rate (FDR) [75] was applied to control the Type I error and to know which predictors remained significant after correction. Second, predictors with an FDR-corrected significance p-value below .10 were selected and a backward stepwise binary logistic regression model was conducted. Predictors from this model with p-values below .05 were used to calculate the area under the ROC curve (AUC), which indicated the proportion of correct predictions of warn- ing signs of victimization achieved with the selected predictors. AUC values above .75 are usu- ally considered as indicating a good prediction capability. This final model included as PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 9 / 19 Warning signs of preschool victimization covariates parents’ and teachers’ answers to item 19 of the SDQ (“Picked on or bullied by other children”) at age 3 to adjust the odds-ratios of victimization at baseline. The predictive capability of the model was calculated through Pseudo-R2 [76] and translated in terms of ordi- nary R2 using the same author’s approach. Since no cross-validation weighted commands have been developed for Stata, the final pre- dictive model was validated using a repeated random sub-sampling cross-validation procedure [77], which produces a summary measure of loss of prediction. The Hosmer-Lemeshow goodness-of-fit test was verified for each logistic model and the absence of outliers and influential observations was also confirmed [78]. Results A total of 602 children were available at age 4 (96.8% from the 622 children starting the study at age 3) and 577 (92.8%) remained at age 5. Participants and drop-outs at age 5 were statisti- cally equal in sex (p = .297), but participants had a higher socioeconomic status (SES) than drop-outs (p = .002). Parents’ information was obtained from mothers (67.6%), fathers (7.6%) or both (24.8%), with a mean age of 37.6 years. Only 5 out of 577 teachers were male. Table 3 shows the demographic characteristics of participants for the total sample and separately by absence/presence of victimization warning signs. Prevalence of warning signs of peer victimization According to the answers given to SDQ-item 19 (“Picked on or bullied by other children”) by the 577 participants remaining in the study at age 5, and applying our definition of warning signs of peer victimization (if answers were 1 or 2), the parents reported warning signs at one follow-up (4 or 5 years old) for 15.9% of preschoolers and at both follow-ups for 3.0% of pre- schoolers. Teachers reported comparable results, with 16.4% of children presenting warning signs at only one age and 1.4% at 4 and 5 years old. To minimize lack of sensitivity, a child was considered to display warning signs of peer victimization if parents or teachers reported the item positively at both ages 4 and 5. On applying this definition, 24 children (4.16%, 95% CI [2.68, 6.13]) were classified as displaying warning signs of peer victimization. Chi-square tests Table 3. Demographics descriptive for total sample at age 3 and grouped by warning signs of peer victimization at ages 4–5. Total sample Warning signs = No Warning signs = Yes χ2 or Fisher exact’s test N = 577 N = 553 N = 24 N (%) N (%) N (%) p Sex .993 Girls 288 (49.9%) 276 (49.9%) 12 (50.0%) Boys 289 (50.1%) 277 (50.1%) 12 (50.0%) Ethnic group .652 Caucasian 518 (89.8%) 495 (89.5%) 23 (95.8%) Hispanic-American 33 (5.7%) 33 (6.0%) 0 (0%) Other 26 (4.5%) 25 (4.5%) 1 (4.2%) Socioeconomic status .013 High 192 (33.3%) 187 (33.8%) 5 (20.8%) Medium-High 177 (30.7%) 171 (30.9%) 6 (25.0%) Medium 86 (14.9%) 85 (15.4%) 1 (4.2%) Medium-Low 93 (16.1%) 84 (15.2%) 9 (37.5%) Low 29 (5.0%) 26 (4.7%) 3 (12.5%) https://doi.org/10.1371/journal.pone.0221580.t003 PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 10 / 19 Warning signs of preschool victimization revealed that there were no differences by sex or ethnicity in the prevalence of warning signs of peer victimization, but the risk increased as SES decreased (Table 3). Initial selection of risk factors of warning signs of peer victimization at 4 to 5 years old Table 1 presents descriptive statistics for the individual characteristic predictors (demograph- ics, dimensional psychopathology, executive functioning, aggressive behavior and tempera- ment) for children with and without warning signs of peer victimization separately; the last three columns show the results of the first stage of the modelling procedure, consisting of the estimation of several independent logistic regression models to predict warning signs of peer victimization at 4 to 5 years old (one model for each set of predictors: demographics, dimen- sional psychopathology, etc.). Four individual attributes showed significant odds-ratios: low or medium-low SES, CBCL-internalizing, CBCL-developmental problems and BRIEF-P emo- tional control difficulties. After applying the FDR correction to control the increment of the type I error due to the high number of statistical tests performed, two predictors remained sig- nificant at .10 value: SES and BRIEF-P emotional control. Similarly, Table 2 shows descriptive statistics for the family and early predictors (pregnancy, early development, history of abuse, parenting practices and maternal psychopathology). The last three columns show the odds ratios obtained from the first phase of the modelling strategy, estimating independent logistic regression models to predict warning signs of victimization at 4 to 5 years old from each set of predictors (pregnancy, early development, etc.). Five signifi- cant odds ratios were found: economic problems during pregnancy, child’s early difficulties in making friends, a history of childhood abuse among relatives, lower norms and higher ASR- somatic problems as reported by the mother. Statistical significance remained below .10 after FDR correction for three predictors: difficulties in making friends, childhood abuse among rel- atives and lower norms. Final predictive model with risk factors of warning signs of peer victimization at ages 4 to 5 The five predictors with FDR-corrected p-values under .10 (two individual and three family predictors) were selected as the maximum logistic regression model in the second stage of the modelling procedure. To select the significant predictors from these five a backward stepwise regression was applied, which retained four significant predictors (Table 4). This final model Table 4. Final logistic regression model with selected predictors at age 3 of warning signs of peer victimization at ages 4–5. Final model–Pseudo R2 = .22 (n = 565) Logistic regression� OR CI 95% OR Socioeconomic status (low or medium-low) 5.39 1.98; 14.66 BRIEF-P emotional control 1.15 1.04; 1.26 Difficulty in making friends (yes) 4.63 1.35; 15.83 APQ-Pr-Norms 0.80 0.71; 0.91 Note: � Adjusted by parents and teachers’ answers to SDQ item 19 (victimization at 3 years-old); in bold, results for predictors with p < .05 SDQ: Strengths and Difficulties Questionnaire; BRIEF-P: Behavior Rating Inventory of Executive Functioning- Preschool version; APQ-Pr: Alabama Parenting Questionnaire for Preschool https://doi.org/10.1371/journal.pone.0221580.t004 PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 11 / 19 Warning signs of preschool victimization Fig 2. ROC curve and AUC for the predictive final model of warning signs of peer victimization at 4–5 years-old. https://doi.org/10.1371/journal.pone.0221580.g002 was adjusted by the baseline presence/absence of item 19 reported by both parents and teach- ers. The resulting odds ratios show that the risk of presence of warning signs of peer victimiza- tion at 4 to 5 years old increased for children with a low or medium-low SES (p = .001), higher scores in BRIEF-P emotional control (p = .004), difficulties in making friends (p = .015) and parenting practices characterized by lower scores in APQ norms (p < .001). This final multiple model, which included four terms plus the two adjusting terms, obtained a Pseudo-R2 = .22, which according to [76] is approximately equivalent to an R2 = .50 of a lin- ear regression. Fig 2 shows the corresponding ROC curve, which reflects the accuracy of these four variables to predict the presence of warning signs of victimization. The AUC = .78 (95% CI [.75, .81]) is statistically better than a classification made by chance (p < .001) and could be considered as almost excellent [74]. Generalizability of the predictive model could be consid- ered acceptable as loss of prediction obtained through repeated random sub-sampling cross- validation was only .044 in terms of AUC. Discussion The purpose of this study was to report on the prevalence and risk factors of warning signs of preschool victimization as assessed by the SDQ, a widely used screening questionnaire reported by parents and teachers. According to the endorsement of item 19 (“Picked on or bullied by other children”), 4.2% of 4- to 5-year-old in-school preschoolers suffered persistent victimization. Although few studies have reported the prevalence of warning signs of victimi- zation in preschoolers, our estimation of prevalence (4.2%) is in line with that reported by PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 12 / 19 Warning signs of preschool victimization previous studies [10,27,79] for children of similar ages in other cultures using one item of a questionnaire for assessing the rates of bullying. Our assessment of victimization was also done with one item of a widely used instrument, the SDQ. Given that aggressive behavior tolerance at any age should be near 0, our results indicating a prevalence of warning signs of victimization in preschoolers equal to 4.2% showed that from preschool age there was a considerable number of children persistently exposed to being picked on or bullied by other children. The adverse consequences of preschool victimization are manifested in children and adolescents in a deteriorated mental health (depression, anxi- ety, suicidal ideation and suicide attempts) and in the increase of psychosocial problems [80]. A variety of individual and family factors at age 3 increased the risk of parents and teachers reporting persistent victimization: pertaining to a low socioeconomic status family, problems in emotional control, early difficulties in making friends and parental practices characterized by lower norms. These variables explained almost 50% of the variance of parents and teachers reporting victimization with a near to excellent ability to discriminate between the groups with and without warning signs of potential victimization. Low SES at age 3 was more frequent in the at-risk group. Several studies showed that being involved in peer victimization either as victim or perpetrator is more frequent in children and adolescents with a lower SES [10,51]. Families with a low SES might have influenced children’s involvement in victimization in several ways. For instance, they tend to have a lower participa- tion in social activities [81]. In low SES families, parents tend to have less involvement in their children’s education, practicing low levels of academic socialization with them [82]. The edu- cational level of parents was included in the family SES calculation because education not only influences economic status, but also reflects non-economic social characteristics, such as intel- ligence, knowledge, social norms and values, literacy and competence in problem solving [10], all of which are aspects that could be related to child raising behavior and, consequently, to children’s limitations in the development of social skills and coping strategies [10,51]. Families living with this adversity would need victimization surveillance. Difficulties in emotional control reported by teachers was found to increase the risk of parents and teachers reporting that the child was being picked on or bullied. In other words, children who overreact to small problems, are explosive, have outbursts for little reason, are angry or tearful or react very strongly to situations, are targets for victimization. This result is consistent with previous studies, among them a methodologically similar study with parents and teachers as informant and children from 3 to 5 years showing a high association between emotion dysregulation and higher rates of victimization [83]. This finding agrees with predic- tions derived from the social process model [84] because emotional dysregulation may cause peers isolation, negative judgments and consequently increase the risk for victimization. This path from emotional control to victimization through social processes was also evidenced in clinical samples of children with a diagnose of attention-deficit/hyperactivity disorder [85] and children with early manifestations of oppositional defiant disorder [46]. Both disorders are characterized by high emotional reactivity, demonstrating that these disorders at preschool age predicted children’s risk of bullying involvement in the first years of elementary school. Children who show high emotional reactivity at preschool entry may therefore require special vigilance at school. Difficulty in making friends was another strong predictor of parents and teachers reporting that the child was victimized. From early childhood, social competence is an important protec- tive factor for children’s adjustment, helping them to engage in shared activities, adapt to dif- ferent situations, develop friendships, respect turns, regulate emotionality, collaborate, show prosocial behavior, acquire a good reputation among peers and be accepted by them [86]. Social competence is thus a solid base for building good relationships and when this is not well PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 13 / 19 Warning signs of preschool victimization developed the child is at risk of being isolated and isolated children are also a target for victimi- zation [87]. Regarding parenting practices, our results are in line with previous studies supporting that poor parental supervision, harsh parental discipline, low parental involvement with the child and marital discord are associated with childhood aggression and bullying [88]. Relatively little is known, however, about how this influence operates [89]. A low parental educational level in social norms may interfere with proper socialization as the child has a deficit in references about how to behave with others and this has also been related to victimization in general pop- ulation [90]. Limitations This study has some limitations that must be highlighted. We did not include information reported from the child because the young age of the participants and the methods used ham- pered their participation. In older children, parents are usually unaware of peer victimization involvement, in part because young people do not always divulge their experiences with victimization to their parents [91], whereas parental follow-up is tighter with preschoolers. Having no information directly from the child, however, could mean that some cases of vic- timization did not come to light. A second limitation is that a high percentage of the participant attrition in the follow-up at age 5 was among low socioeconomic status families. This could have resulted in a slight under- estimation of the prevalence of victimization. Another limitation is in reference to the social-ecological model, because we have not taken into account the community contexts in which social relationships are embedded. The class- room environment had a great influence on the amount of victimization reported by peers, indicating that there is something in the class context that potentiates or inhibits bullying [92]. On the other hand, this study also has several strengths. Peer victimization was assessed using multiple informants (parents and teachers) and considers risk factors from multiple domains (individual and family). Last, much of the previous work on victims has been con- ducted with older children and there are only a limited number of studies that focus on the preschool period. Conclusions Peer-victimization starts at preschool ages, so early intervention in the preschool context is crucial if we are to break the cycle of victimization in later childhood [34]. Since both individ- ual and family risk factors of different nature are present in the victims, to stop the victimiza- tion from escalating requires a broad and multidisciplinary approach. Implications for school health School is a context where peer victimization may occur, and this markedly alters community mental health. The development of programs designed to prevent its escalation, reduce its neg- ative impact on children’s development and promote adjustment are needed. Early identifica- tion of both the problem itself, using screening questionnaires for detecting warning signs, and of the problems associated with victimization dynamics may enable closer monitoring of the risk groups as a preventative measure. The prevention of victimization and its conse- quences can be improved by focusing on risk groups at school in early life. Our results suggest that victimization intervention programs should consider the role of the victims’ personal characteristics in increasing vulnerability to victimization and the importance of including families in school-based intervention programs aimed at reducing the difficulties experienced PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 14 / 19 Warning signs of preschool victimization by victimized children. In summary, research identifying the risk factors for victimization helps to target interventions to deal with the key factors contributing to the problem, thus maximizing their effectiveness [92]. Acknowledgments We would like to thank the participating schools and families. Author Contributions Conceptualization: Meritxell Fernández, Núria de la Osa, Lourdes Ezpeleta. Data curation: Jose-Blas Navarro, Eva Penelo. Formal analysis: Jose-Blas Navarro, Eva Penelo. Funding acquisition: Lourdes Ezpeleta. Investigation: Meritxell Fernández, Núria de la Osa. Methodology: Jose-Blas Navarro, Lourdes Ezpeleta. Project administration: Lourdes Ezpeleta. Writing – original draft: Jose-Blas Navarro, Meritxell Fernández, Eva Penelo, Lourdes Ezpeleta. Writing – review & editing: Jose-Blas Navarro, Meritxell Fernández, Núria de la Osa, Eva Penelo, Lourdes Ezpeleta. References 1. Cooley JL, Fite PJ, Pederson CA. Bidirectional associations between peer victimization and functions of aggression in middle childhood: Further evaluation across informants and academic years. J Abnorm Child Psychol. 2017; https://doi.org/10.1007/s10802-017-0283-8 PMID: 28217827 2. Salmivalli C. Bullying and the peer group: A review. Aggress Violent Behav. 2010; 15: 112–120. https:// doi.org/10.1016/j.avb.2009.08.007 3. Copeland WE, Wolke D, Angold A, Costello EJ. Adult Psychiatric Outcomes of Bullying and Being Bul- lied by Peers in Childhood and Adolescence. JAMA Psychiatry. 2013; 70: 419. https://doi.org/10.1001/ jamapsychiatry.2013.504 PMID: 23426798 4. Takizawa R, Maughan B, Arseneault L. Adult health outcomes of childhood bullying victimization: evi- dence from a five-decade longitudinal British birth cohort. Am J Psychiatry. 2014; 171: 777–84. https:// doi.org/10.1176/appi.ajp.2014.13101401 PMID: 24743774 5. Kochenderfer BJ, Ladd GW. Peer victimization: cause or consequence of school maladjustment? Child Dev. 1996; 67: 1305–17. http://www.ncbi.nlm.nih.gov/pubmed/8890485 PMID: 8890485 6. Bowes L, Maughan B, Caspi A, Moffitt TE, Arseneault L. Families promote emotional and behavioural resilience to bullying: evidence of an environmental effect. J Child Psychol Psychiatry. 2010; 51: 809– 817. https://doi.org/10.1111/j.1469-7610.2010.02216.x PMID: 20132419 7. Barker ED, Arsenault L, Brendgen M, Fontaine N, Maughan B. Joint Development of Bullying and Vic- timization in Adolescence. J Am Acad Child Adolesc Psychiatry. 2008; 47: 1030–1038. https://doi.org/ 10.1097/CHI.ObO13e31817eec98 PMID: 18665001 8. Juvonen J, Graham S. Bullying in schools: the power of bullies and the plight of victims. Annu Rev Psy- chol. 2014; 65: 159–85. https://doi.org/10.1146/annurev-psych-010213-115030 PMID: 23937767 9. Herba CM, Ferdinand RF, Stijnen T, Veenstra R, Oldehinkel AJ, Ormel J, et al. Victimization and suicide ideation in the TRAILS study: specific vulnerabilities of victims. J Child Psychol Psychiatry. 2008; 49: 867–876. https://doi.org/10.1111/j.1469-7610.2008.01900.x PMID: 18492041 10. Jansen PW, Verlinden M, Berkel AD-V, Mieloo C, Van Der Ende J, Veenstra R, et al. Prevalence of bul- lying and victimization among children in early elementary school: Do family and school neighbourhood socioeconomic status matter? BMC Public Health. 2012; 12: 494. https://doi.org/10.1186/1471-2458- 12-494 PMID: 22747880 PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 15 / 19 Warning signs of preschool victimization 11. Crick NR, Bigbee MA. Relational and overt forms of peer victimization: A multiinformant approach. J Consult Clin Psychol. 1998; 66: 337–347. https://doi.org/10.1037//0022-006x.66.2.337 PMID: 9583337 12. Nansel TR, Overpeck M, Pilla RS, Ruan WJ, Simons-Morton B, Scheidt P. Bullying behaviors among US youth—Prevalence and association with psychosocial adjustment. Jama-Journal Am Med Assoc. 2001; 285: 2094–2100. https://doi.org/10.1001/jama.285.16.2094 PMID: 11311098 13. Kaltiala-Heino R, Rimpelä M, Rantanen P, Rimpelä A. Bullying at school-an indicator of adolescents at risk for mental disorders. J Adolesc. 2000; 23: 661–674. https://doi.org/10.1006/jado.2000.0351 PMID: 11161331 14. Arseneault L, Walsh E, Trzesniewski K, Newcombe R, Caspi A, Moffitt TE. Bullying victimization uniquely contributes to adjustment problems in young children: a nationally representative cohort study. Pediatrics. 2006; 118: 130–138. https://doi.org/10.1542/peds.2005-2388 PMID: 16818558 15. Strøm IF, Aakvaag HF, Birkeland MS, Felix E, Thoresen S. The mediating role of shame in the relation- ship between childhood bullying victimization and adult psychosocial adjustment. Eur J Psychotrauma- tol. 2018; 9: 1418570. https://doi.org/10.1080/20008198.2017.1418570 PMID: 29372013 16. Laditka JN, Laditka SB. Adverse Childhood Circumstances and Functional Status Throughout Adult Life. J Aging Health. 2018; 30: 1347–1368. https://doi.org/10.1177/0898264317715976 PMID: 28643582 17. Sigurdson JF, Wallander J, Sund AM. Is involvement in school bullying associated with general health and psychosocial adjustment outcomes in adulthood? Child Abuse Negl. 2014; 38: 1607–17. https:// doi.org/10.1016/j.chiabu.2014.06.001 PMID: 24972719 18. Vlachou M, Andreou E, Botsoglou K, Didaskalou E. Bully/victim problems among preschool children: A review of current research evidence. Educ Psychol Rev. 2011; 23: 329–358. https://doi.org/10.1007/ s10648-011-9153-z 19. Hanish LD, Ryan P, Martin CL, Fabes RA. The social context of young children’s peer victimization. Soc Dev. 2005; 14: 2–19. https://doi.org/10.1111/j.1467-9507.2005.00288.x PMID: 16493454 20. Beauchesne MA, Farquharson PE, Beauchemin P. Buddies Not Bullies: an approach to bullying pre- vention education in preschool children, their families and the community. Nurs Child Young People. 2017; 29: 41–45. https://doi.org/10.7748/ncyp.2017.e843 PMID: 28395634 21. Huang H, Liu Y, Chen Y. Preservice Preschool Teachers’ Responses to Bullying Scenarios: The Roles of Years of Study and Empathy. Front Psychol. 2018; 9: 175. https://doi.org/10.3389/fpsyg.2018.00175 PMID: 29515492 22. Björkqvist K. Different Names, Same Issue. Soc Dev. John Wiley & Sons, Ltd (10.1111); 2001; 10: 272–274. https://doi.org/10.1111/1467-9507.00164 23. Monks CP, Smith PK, Swettenham J. Psychological correlates of peer victimisation in preschool: social cognitive skills, executive function and attachment profiles. Aggress Behav. John Wiley & Sons, Ltd; 2005; 31: 571–588. https://doi.org/10.1002/ab.20099 24. Rigby K, Smith PK. Is school bullying really on the rise? Soc Psychol Educ An Int J. 2011; 14: 441–455. https://doi.org/10.1007/s11218-011-9158-y 25. Craig W, Harel-Fisch Y, Fogel-Grinvald H, Dostaler S, Hetland J, Simons-Morton B, et al. A cross- national profile of bullying and victimization among adolescents in 40 countries. Int J Public Health. 2009; 54. https://doi.org/10.1007/s00038-009-5413-9 PMID: 19623475 26. Monks CP, Smith PK, Swettenham J. Aggressors, victims, and defenders in preschool: Peer, self-, and teacher reports. Merrill Palmer Q. 2003; 49: 453–469. 27. Ilola A-M, Lempinen L, Huttunen J, Ristkari T, Sourander A. Bullying and victimisation are common in four-year-old children and are associated with somatic symptoms and conduct and peer problems. Acta Paediatr Int J Paediatr. 2016; 105: 522–528. https://doi.org/10.1111/apa.13327 PMID: 26741067 28. Goodman R. The Strengths and Difficulties Questionnaire: A Research Note. J Child Psychol Psychia- try. 1997; 38: 581–586. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x PMID: 9255702 29. Dahlberg A, Ghaderi A, Sarkadi A, Salari R. SDQ in the Hands of Fathers and Preschool Teachers— Psychometric Properties in a Non-clinical Sample of 3–5-Year-Olds. Child Psychiatry Hum Dev. Springer US; 2019; 50: 132–141. https://doi.org/10.1007/s10578-018-0826-4 PMID: 29959588 30. Fält E, Sarkadi A, Fabian H. Exploring Nurses’, Preschool Teachers’ and Parents’ Perspectives on Information Sharing Using SDQ in a Swedish Setting—A Qualitative Study Using Grounded Theory. PLoS One. Public Library of Science; 2017; 12: e0168388. https://doi.org/10.1371/journal.pone. 0168388 PMID: 28076401 31. De Los Reyes A, Henry DB, Tolan PH, Wakschlag LS. Linking informant discrepancies to observed var- iations in young children’s disruptive behavior. J Abnorm Child Psychol. NIH Public Access; 2009; 37: 637–52. https://doi.org/10.1007/s10802-009-9307-3 PMID: 19247829 PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 16 / 19 Warning signs of preschool victimization 32. De Los Reyes A, Kazdin AE. Informant Discrepancies in the Assessment of Childhood Psychopathol- ogy: A Critical Review, Theoretical Framework, and Recommendations for Further Study. Psychol Bull. 2005; 131: 483–509. https://doi.org/10.1037/0033-2909.131.4.483 PMID: 16060799 33. Álvarez-Garcı́a D, Garcı́a T, Núñez JC. Predictors of school bullying perpetration in adolescence: A sys- tematic review. Aggress Violent Behav. Pergamon; 2015; 23: 126–136. https://doi.org/10.1016/J.AVB. 2015.05.007 34. Bowes L, Maughan B, Ball H, Shakoor S, Ouellet-Morin I, Caspi A, et al. Chronic bullying victimization across school transitions: the role of genetic and environmental influences. Dev Psychopathol. 2013; 25: 333–346. https://doi.org/10.1017/S0954579412001095 PMID: 23627948 35. Espelage D, Swearer S. Research on School Bullying and Victimization: What Have We Learned and Where Do We Go from Here? School Psych Rev. 2003; 32: 365–383. 36. Bronfenbrenner U. The Ecology of Human Development—Experiments by Nature and Design [Inter- net]. Harvard University Press; 1979. https://doi.org/10.1080/00131728109336000 37. Garbarino J, Crouter A. Defining the Community Context for Parent-Child Relations: The Correlates of Child Maltreatment. Child Dev. WileySociety for Research in Child Development; 1978; 49: 604–616. https://doi.org/10.2307/1128227 38. Garbarino J. Adolescent development : an ecological perspective [Internet]. C.E. Merrill Pub. Co; 1985. https://books.google.es/books/about/Adolescent_development.html?id=6JQEAQAAIAAJ&redir_esc=y 39. Dahlberg LL, Krug EG. Violence a global public health problem. Cien Saude Colet. ABRASCO—Asso- ciação Brasileira de Saúde Coletiva; 2006; 11: 277–292. https://doi.org/10.1590/S1413- 81232006000200007 40. Dahlberg L, Krug E. Violence: a global public health problem. In: Krug E, Dahlberg L, Mercy J, Zwi A, Lozano R, editors. World Report on Violence and Health. John Wiley & Sons, Ltd; 2002. https://doi.org/ 10.1002/casp.740 41. Hodges E, Malone M, Perry D. Individual risk and social risk as interacting determinants of victimization in the peer group. Dev Psychol. 1997; 33: 1032–1039. PMID: 9383625 42. Carney AG, Merrell KW. Bullying in Schools. Perspectives on understanding and preventing an interna- tional problem. Sch Psychol Int. 2001; 22: 364–382. https://doi.org/10.1177/0143034301223011 43. Griffin RS, Gross AM. Childhood bullying: Current empirical findings and future directions for research. Aggress Violent Behav. 2004; 9: 379–400. https://doi.org/10.1016/S1359-1789(03)00033-8 44. Frizzo MN, Bisol LE, Lara DR. Bullying victimization is associated with dysfunctional emotional traits and affective temperaments. J Affect Disord. 2013; 148: 48–52. https://doi.org/10.1016/j.jad.2012.11. 046 PMID: 23245468 45. Hodges E, Perry D. Personal and interpersonal antecedents and consequences of victimization by peers. J Pers Soc Psychol. 1999; 76: 677–85. http://www.ncbi.nlm.nih.gov/pubmed/10234851 PMID: 10234851 46. Verlinden M, Jansen PW, Veenstra R, Jaddoe VW V, Hofman A, Verhulst FC, et al. Preschool Atten- tion-Deficit/Hyperactivity and Oppositional Defiant Problems as Antecedents of School Bullying. J Am Acad Child Adolesc Psychiatry. Elsevier Inc.; 2015; 54: 571–579. https://doi.org/10.1016/j.jaac.2015. 05.002 PMID: 26088662 47. Medeiros W, Torro-Alves N, Malloy-Diniz LF, Minervino CM. Executive Functions in Children Who Experience Bullying Situations. Front Psychol. 2016; 7. https://doi.org/10.3389/fpsyg.2016.01197 PMID: 27616998 48. Bradshaw CP, Sawyer AL, O’Brennan LM. A Social Disorganization Perspective on Bullying-Related Attitudes and Behaviors: The Influence of School Context. Am J Community Psychol. 2009; 43: 204– 220. https://doi.org/10.1007/s10464-009-9240-1 PMID: 19333749 49. Sawyer AL, Bradshaw CP, O’Brennan LM. Examining Ethnic, Gender, and Developmental Differences in the Way Children Report Being a Victim of “Bullying” on Self-Report Measures. J Adolesc Heal. 2008; 43: 106–114. https://doi.org/10.1016/j.jadohealth.2007.12.011 PMID: 18639783 50. Wolke D. Bullying involvement in primary school and common health problems. Arch Dis Child. 2001; 85: 197–201. https://doi.org/10.1136/adc.85.3.197 PMID: 11517098 51. Perren S, Stadelmann S, von Klitzing K. Child and family characteristics as risk factors for peer victimi- zation in kindergarten. Schweizerische Zeitschrift fur Bild. 2009; 31: 13–32. 52. Burk LR, Park J, Armstrong JM, Klein MH, Goldsmith HH, Zahn-Waxler C, et al. Identification of early child and family risk factors for aggressive victim status in first grade. J Abnorm Child Psychol. 2008; 36: 513–526. https://doi.org/10.1007/s10802-007-9196-2 PMID: 18092191 53. Mohr A. Family variables associated with peer victimization: Does family violence enhance the probabil- ity of being victimized by peers? Swiss J Psychol. 2006; 65: 107–116. https://doi.org/10.1024/1421- 0185.65.2.107 PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 17 / 19 Warning signs of preschool victimization 54. Due P, Damsgaard MT, Lund R, Holstein BE. Is bullying equally harmful for rich and poor children?: a study of bullying and depression from age 15 to 27. Eur J Public Health. 2009; 19: 464–469. https://doi. org/10.1093/eurpub/ckp099 PMID: 19587227 55. Lereya ST, Wolke D. Prenatal family adversity and maternal mental health and vulnerability to peer victi- misation at school. J Child Psychol Psychiatry. 2013; 54: 644–652. https://doi.org/10.1111/jcpp.12012 PMID: 23121554 56. Pears KC, Capaldi DM. Intergenerational transmission of abuse: a two-generational prospective study of an at-risk sample. Child Abuse Negl. 2001; 25: 1439–1461. https://doi.org/10.1016/S0145-2134(01) 00286-1 PMID: 11766010 57. Goodman A, Joyce R, Smith JP. The long shadow cast by childhood physical and mental problems on adult life. Proc Natl Acad Sci U S A. 2011; 108: 6032–6037. https://doi.org/10.1073/pnas.1016970108 PMID: 21444801 58. Vessey J, Strout TD, DiFazio RL, Walker A. Measuring the youth bullying experience: A systematic review of the psychometric properties of available instruments. J Sch Health. 2014; 84: 819–843. https://doi.org/10.1111/josh.12210 PMID: 25388599 59. Ezpeleta L, de la Osa N, Doménech JM. Prevalence of DSM-IV disorders, comorbidity and impairment in 3-year-old Spanish preschoolers. Soc Psychiatry Psychiatr Epidemiol. 2014; 49: 145–155. https:// doi.org/10.1007/s00127-013-0683-1 PMID: 23595297 60. Merikangas KR, He J-P, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of men- tal disorders in U.S. adolescents: results from the National Comorbidity Survey Replication—Adoles- cent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. NIH Public Access; 2010; 49: 980–9. https://doi.org/10.1016/j.jaac.2010.05.017 PMID: 20855043 61. Ezpeleta L, Granero R, de la Osa N, Penelo E, Doménech JM. Psychometric properties of the Strengths and Difficulties Questionnaire3-4 in 3 year-old preschoolers. Compr Psychiatry. 2013; 54: 282–291. https://doi.org/10.1016/j.comppsych.2012.07.009 PMID: 22999518 62. Achenbach TM, Rescorla LA. Manual for the ASEBA preschool-age forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families; 2000. 63. Gioia GA, Espy KA, Isquith PK. Behavior Rating Inventory of Executive Function—Preschool version. Lutz, FL: Psychological Assessment Resources; 2003. 64. Ezpeleta L, Granero R, Penelo E, de la Osa N, Domènech JM. Behavior Rating Inventory of Executive Functioning–Preschool (BRIEF-P) Applied to Teachers. J Atten Disord. 2015; 19: 476–488. https://doi. org/10.1177/1087054712466439 PMID: 23264366 65. Halperin JM, McKay KE. Children’s Aggression Scale. Lutz, FL: Psychological Assessment Resources; 2008. 66. Putnam SP, Rothbart MK. Development of short and very short forms of the Children’s Behavior Ques- tionnaire. J Pers Assess. 2006; 87: 103–113. 67. de la Osa N, Granero R, Penelo E, Domènech JM, Ezpeleta L. The Short and Very Short Forms of the Children’s Behavior Questionnaire in a Community Sample of Preschoolers. Assessment. 2014; 21: 463–476. https://doi.org/10.1177/1073191113508809 PMID: 24235176 68. Unitat d’Epidemiologia i de Diagnòstic en Psicopatologia del Desenvolupament. Protocolo de Factores de Riesgo. Versión para preescolares. Barcelona: Universitat Autònoma de Barcelona, Departament de Psicologia Clı́nica i de la Salut; 2009. 69. Hollingshead AB. Four factor index of social status [Internet]. New Haven, CT: Unpublished manu- script, Yale University, Department of Sociology; 1975. http://www.yale-university.com/sociology/ faculty/docs/hollingshead_socStat4factor.pdf 70. Guillamón N. Fiabilidad entre entrevistadores del Protocolo de Factores de Riesgo-Versión niños. [Inter-rater fiability of the risk factors protocol-Children version]. 1999. 71. de la Osa N, Granero R, Penelo E, Doménech JM, Ezpeleta L. Psychometric properties of the Alabama Parenting Questionnaire–Preschool revision (APQ-Pr) in 3 year-old Spanish preschoolers. J Child Fam Stud. 2014; 23: 776–784. https://doi.org/10.1007/s10826-013-9730-5 72. Meunier J-C, Roskam I. Validation of the preschool and primary school form of a questionnaire assess- ing parents’ childrearing behavior. J Clin Child Adolesc Psychol. 2009; 38: 166–75. https://doi.org/10. 1080/15374410802575370 PMID: 19130365 73. Achenbach TM, Rescorla LA. Manual for the ASEBA adult forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families; 2003. 74. Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. 3rd Ed. New Jersey: John Wiley & Sons; 2013. 75. Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to mul- tiple testing. J R Stat Soc Ser B. 1995; 7: 289–300. PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 18 / 19 Warning signs of preschool victimization 76. McFadden D. Conditional logit analysis of qualitative choice behavior. In: Zarembka P, editor. Frontiers in Econometrics. New York: Academic Press.; 1973. pp. 105–142. 77. Dubitzky W, Granzow M, Berrar D. Fundamentals of data mining in genomics and proteomics. New York: Springer; 2007. 78. Kleinbaum DG, Klein M. Logistic Regression. A self-learning text. 3rd ed. New York: Springer-Verlag; 2010. 79. Øksendal E, Brandlistuen RE, Holte A, Wang MV. Peer-Victimization of Young Children With Develop- mental and Behavioral Difficulties—A Population-Based Study. J Pediatr Psychol. 2019; 44: 589–600. https://doi.org/10.1093/jpepsy/jsy112 PMID: 30816959 80. Moore SE, Norman RE, Suetani S, Thomas HJ, Sly PD, Scott JG. Consequences of bullying victimiza- tion in childhood and adolescence: A systematic review and meta-analysis. World J psychiatry. Baishi- deng Publishing Group Inc; 2017; 7: 60–76. https://doi.org/10.5498/wjp.v7.i1.60 PMID: 28401049 81. Stevens V, De Bourdeaudhuij I, Van Oost P. Relationship of the Family Environment to Children’s Involvement in Bully/Victim Problems at School. J Youth Adolesc. Kluwer Academic Publishers-Plenum Publishers; 2002; 31: 419–428. https://doi.org/10.1023/A:1020207003027 82. Carolan B V., Wasserman SJ. Does Parenting Style Matter? Concerted Cultivation, Educational Expec- tations, and the Transmission of Educational Advantage. Sociol Perspect. SAGE PublicationsSage CA: Los Angeles, CA; 2015; 58: 168–186. https://doi.org/10.1177/0731121414562967 83. Godleski SA, Kamper KE, Ostrov JM, Hart EJ, Blakely-McClure SJ. Peer Victimization and Peer Rejec- tion During Early Childhood. J Clin Child Adolesc Psychol. 2015; 44: 380–392. https://doi.org/10.1080/ 15374416.2014.940622 PMID: 25133659 84. Boivin M, Hymel S, Hodges EVE. Toward a process view of peer rejection and harassment. In: Juvonen J, Graham S, editors. Peer harassment in school: The plight of the vulnerable and victimized. New York, NY: US: The Guilford Press; 2001. pp. 265–289. 85. Hoza B, Mrug S, Gerdes AC, Hinshaw SP, Bukowski WM, Gold JA, et al. What Aspects of Peer Rela- tionships Are Impaired in Children With Attention-Deficit/Hyperactivity Disorder? J Consult Clin Psychol. 2005; 73: 411–423. https://doi.org/10.1037/0022-006X.73.3.411 PMID: 15982139 86. Denham SA, Blair KA, Demulder E, Levitas J, Sawyer K, Auerbach-Major S, et al. Preschool Emotional Competence: Pathway to Social Competence? Child Dev. 2003; 74: 238–256. PMID: 12625448 87. Perren S, Alsaker FD. Social behavior and peer relationships of victims, bully-victims, and bullies in kin- dergarten. J Child Psychol Psychiatry. 2006; 47: 45–57. https://doi.org/10.1111/j.1469-7610.2005. 01445.x PMID: 16405640 88. Bibou-Nakou I, Tsiantis J, Assimopoulos H, Chatzilambou P. Bullying/victimization from a family per- spective: a qualitative study of secondary school students’ views. Eur J Psychol Educ. 2013; 28: 53–71. https://doi.org/10.1007/s10212-011-0101-6 89. Jiménez TI, Musitu G, Ramos MJ, Murgui S. Community involvement and victimization at school: An analysis through family, personal and social adjustment. J Community Psychol. 2009; 37: 959–974. https://doi.org/10.1002/jcop.20342 90. Baldry AC, Farrington DP. Protective factors as moderators of risk factors in adolescence bullying. Soc Psychol Educ. 2005; 8: 263–284. https://doi.org/10.1007/s11218-005-5866-5 91. Houndoumadi A, Pateraki L. Bullying and bullies in Greek elementary schools: Pupils’ attitudes and teachers’/parents’ awareness. Educ Rev. 2001; 53: 19–26. 92. Saarento S, Kärnä A, Hodges E, Salmivalli C. Student-, classroom-, and school-level risk factors for victimization. J Sch Psychol. 2013; 51: 421–434. https://doi.org/10.1016/j.jsp.2013.02.002 PMID: 23816233 PLOS ONE | https://doi.org/10.1371/journal.pone.0221580 August 23, 2019 19 / 19