Psychosocial Intervention (2019) 28(1) 1-10 Vol. 27. No. 1, April 2018 ISSN: 1132-0559 Psychosocial Intervention Psychosocial Intervention Editor Enrique Gracia Associate Editors Fernando Chacón Manuel García-Ramírez Marisol Lila Gonzalo Musitu Douglas D. Perkins h t t p s : / / j o u r n a l s. c o p m a d r i d. o r g / p i Consejo General de la Psicología ESPAÑA Evaluation of a Parenting Training Program, “Limits”, in a Juvenile Justice Service: Results and Challenges Màrius Martínez-Muñoz, Laura Arnau, and Marta Sabaté Universitat Autònoma de Barcelona, Bellaterra (Cerdanyola del Vallès), Barcelona, Spain ARTICLE INFO A B S T R A C T Article history: The aim of the study is to evaluate the results of “Limits”, a parenting training program which was implemented in Received 27 October 2017 a juvenile justice service by professionals from the Department of Justice. A controlled before and after design was Accepted 16 April 2018 undertaken to measure the effects of the program on the use of parental practices and parents’ perception of self-efficacy. Available online 19 July 2018 Fifty-nine families with adolescents involved in the juvenile justice system participated either in an intervention group (n = 42 families) or in a waiting list group (n = 17 families). In order to triangulate the information, professionals and the target adolescents were also asked about the improvement of the parents in their parenting practices. A generalized Keywords: linear model was used to compare the intervention and comparison groups. The findings demonstrate that the program Program evaluation Juvenile justice had a positive impact on the intervention group with regards to the transfer of practices to their real life, especially those Family intervention related to communication and family relationships. However, no significant changes were observed in parents’ perception Self-efficacy of self-efficacy. The results are geared towards the improvement of future research evaluations using longitudinal designs. Parental practices La evaluación de un programa de formación parental, Límites, en un servicio de justicia juvenil: resultados y retos R E S U M E N Palabras clave: El objetivo de este estudio es evaluar los resultados del programa de entrenamiento de habilidades parentales “Limits” im- Evaluación de programas plementado en un servicio de justicia juvenil por profesionales del Departamento de Justicia. Se aplicó un diseño cuasi ex- Justicia juvenil Intervención familiar perimental pre-post test con grupo control para medir los efectos del programa en la utilización de las prácticas parentales y Autoeficacia la percepción de autoeficacia de las familias. Participaron 59 familias con adolescentes dentro del circuito de justicia juvenil, Prácticas parentales ya fuera formando parte del grupo de intervención (n = 42 familias) o del grupo en lista de espera para entrar en el programa (n = 17 familias). Para triangular la información, se preguntó tanto a los profesionales participantes como los propios adoles- centes por su percepción de mejora en relación con las prácticas parentales aplicadas por las familias en el ámbito cotidiano. Se aplicó un modelo lineal general para comparar el grupo intervención y el grupo comparación. Los resultados demuestran que el programa tuvo un impacto positivo en el grupo de intervención en lo referente a la transferencia de las prácticas pa- rentales a la vida real, especialmente en aquéllas relacionadas con la comunicación y las relaciones familiares. Sin embargo, no se observaron cambios significativos en la percepción de autoeficacia por parte de los progenitores. Los hallazgos de este estudio están orientados hacia la mejora de futuras investigaciones utilizando diseños de evaluación longitudinales. Social changes in the dynamics of family life have led to European set of behaviors based on the best interests of the child (upbringing, Union (EU) governments to become aware of the need to respond to empowerment, non-violence, setting limits), which in turn allow the and support families in developing positive transition and adaptation child to develop fully and achieve better outcomes at home, at school, processes for their internal dynamics (Martínez González & Becedóniz with friends, and in the community. As underlined by Martínez Vázquez, 2009). This resulted in the Committee of Ministers of the and Becedóniz (2009), the EU recommends that positive parenting Member States of the EU dictating Recommendation 19 in support practices be promoted and supported through institutional services of the development of positive parenting (Council of Europe, 2006), and educational programs. The structured training of family members highlighting those activities which promote positive parenting as a has proven to be effective in reducing disruptive behavioral problems Cite this article as: Martínez-Muñoz, M., Arnau, L., & Sabaté, M. (2019). Evaluation of a parenting training program, “Limits”, in a juvenile justice service: Results and challenges. Psychosocial Intervention, 28, 1-10. https://doi.org/10.5093/pi2018a14 Funding: This study was supported by a grant from CEJFE and Àrea de Medi Obert i de la Unitat de Programes d’Innovació de la Direcció General d’Execusió Penal a la Comunitat i de Justícia Juvenil, Department of Justice, Autonomous Government of Catalonia. Correspondence: Laura.arnau@uab.cat (L. Arnau). ISSN:1132-0559/© 2019 Colegio Oficial de Psicólogos de Madrid. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). 2 M. Martínez-Muñoz et al. / Psychosocial Intervention (2019) 28(1) 1-10 in young people (Axberg & Broberg, 2012; Fletcher, Freeman, & (perceiving less and less frequent disruptive behaviors in children) Mathey, 2011), showing that it is possible to modify educational skills but not in the expression of clear expectations or self-efficacy (which and therefore teach family management skills as well as train them. seems to take longer, especially in difficult interaction paths between As Letarte, Normandeau, and Allard (2010) noted, the primary agents parents and children). They also noted that families preferred to of socialization for children and young people with difficulties are participate in group programs rather than in individual therapy. their families, so it is essential to examine which types of intervention Axberg and Broberg (2012) highlighted the positive results when can provide these families with effective support. emotional communication skills, positive interaction with children Martín-Quintana et al. (2009) identified three different generations or young people, and consistency in discipline have been taught. of group programs for promoting parenting skills. First generation Research from the “The Incredible Years” program in Sweden has programs emphasize the quality of parenting patterns in providing a shown positive results in reducing disruptive behavior in contrast supportive and stimulating environment for children’s development. with the group in the waiting list, one year after the completion of Second generation programs, consisting of bidirectional programs, the program. Webster, Gaspar, and Seabra-Santos (2012) showed emphasize the quality of parents’ and children’s interactions in the results of the program when adapted to the Portuguese context daily activities, attachment, empathy, setting limits, and managing – after it had been successfully adapted to UK, Ireland, Norway, inappropriate and/or aggressive behaviors. Third generation Denmark, Finland, and Holland – noting how research has shown that programs seek to improve quality in how the family system functions early intervention is the key to addressing problems and that actions through comprehensive, durable, multidomain, and multicontext during the adolescent period have limited effect, especially when actions. With regard to training parents, the authors refer to three aimed at criminal or aggressive behaviors or secondary risk factors models: academic, managerial, and experiential model. (school failure, absenteeism, and substance abuse). Salari, Ralph, In the international context, Sanders, Kirby, Tellegen, and Day and Sanders’ (2014) evaluation of the Positive Parenting Program (2014) identify several parental training programs, shaped according for Parents of Teenagers reported a decrease in teen disruptive to different formats, theoretical approaches, training methods, behaviors, a reduction in the negative impacts associated with teen intensities, and durations (McConnell, Breitkreuz, & Savage, 2012), difficulties, and a greater improvement in parent-child relationships. including “The Incredible Years” (Webster-Stratton, 1998), “Parent The intervention was also associated with a greater reduction in Management Training - Oregon Model” (PMTO; Forgatch & Patterson, using coercive parenting strategies. Taylor et al. (2015) evaluated The 2010), “Parent-Child Interaction Therapy” (Fernandez & Eyberg, Adolescent Parent Ways program for parents of youths experiencing 2009), “The Triple P-Positive Parenting Program”- especially at levels social behavioral issues, focusing on enhancing parent-adolescent 4 and 5 - (Sanders, 2012), all of which have different formats. In Spain, relationship quality and on the increase of parental knowledge multidisciplinary preventive programs targeting different types of regarding adolescent development, monitoring of adolescents, needs in different contexts have been developed. Methodologies are and reduction of parent and adolescent stress. It also showed often bidirectional, experiential, in-group, active, and participatory. improvements in the overall parent-adolescent relationship. The following programs are worth highlighting in this respect: Simoes, Fonseca, and Anglin (2014) also identified a high level of “Program Guide for the development of competencies” (Ministry of satisfaction among professionals applying these programs; Taylor Social Affairs; Martínez González, 2009) and “Program of educational et al (2015) demonstrated some advantages that result from having interventions and family” (Bartau & De la Caba Collado, 2009), as multiple group sessions with other parents led by trained group second-generation programs, and “Programa de Apoyo Personal y leaders. Parents reported lower levels of perceived stress during familiar” (Martín, Máiquez, Rodrigo, Correa, & Rodríguez, 2004) and post-testing, indicating that the in-person parent training resulted “Aprender Juntos, crecer en família” (Amorós, Balsells, Mateos, Jose in overall improvements in relationship dynamics and well-being for & Vaquero, 2014) as third-generation programs. In Catalonia, the parents. Department of Health and Justice has developed and applied “Limits”, The assessment carried out by Bartau and de la Caba Collado a second-generation program (Larriba, 2010). (2009) in Spain identified improvements in the understanding of Several authors warn that many of the programs have been poorly children’s needs, parenting skills, children’s emotional development evaluated (Letarte et al., 2010; Rodrigo, Martín, Cabrera, & Máiquez, and family relationships, as well as in children’s awareness of parents’ 2009) and a significant proportion of evaluated programs display difficulties in taking care of them. More recently, Martínez-González, significant methodological limitations: very few use control groups Rodríguez-Ruiz, Álvarez-Blanco, and Becedóniz-Vázquez (2016) (Lundahl, Nimer, & Parsons, 2006) and they are all aimed at a very reported that after participating in the Program-Guide with children limited number of families or at a narrow age gap (Sanders, Markie- aged from 1 to 18 years old, participants perceived themselves more Dadds, Rinaldis, Firmani, & Baig, 2007). Also, some studies show competent as parents in all five dimensions analyzed (emotional self- little evidence of sustained deterrent effects through strong research regulation abilities, self-esteem and assertiveness, communication designs and multiple site replications (Mihalic, Fagan, Irwin, Ballard, strategies, conflict resolution strategies and coherent limits, norms & Elliott, 2004); consequently, they have a limited impact at the and consequences strategies). population level (Sanders et al., 2014). As Mulford et al. (2016) underlined, several characteristics Mulford, Redding, and Mendoza (2016) note that there are few moderate the effectiveness of parent-training programs: socio- studies on behavioral parent-training programs aimed at treating economically disadvantaged families, single parent families, and justice-involved juvenile populations. Most of the research on families lacking social support are less likely to demonstrate gains parent-training programs is focused on treating disruptive behavior from these programs. The age of children is also associated with in younger children. Reviews by Letarte et al. (2010), considering effectiveness, being higher when having younger offspring. These several meta-analyses of studies on these programs and their results, authors also noted that there are relatively few studies of behavioral indicate that they showed positive effects on parents’ attitudes and parent training programs aimed at treating justice-involved juvenile behaviors towards their children in families with children with populations. In such cases, results showed beneficial effects on family behavioral problems. These programs also seem to be effective for communication and family relationships but only modest reductions these children. Parents’ sense of competence or self-efficacy has been on offending. one of the factors used in program evaluations (Gilmore & Cuskelly, In short, this article aims to approach an evidence-based assessment 2008), with differences being found between mothers and fathers in of the parent-training program “Limits” developed in Catalonia this issue (Fletcher et al., 2011). Letarte et al. (2010) showed a positive and implemented in several editions since 2007. Particularly, the impact on parental practices and the perception of infant behavior evaluation focuses on the effect of the intervention on parents’ self- Evaluation of a Parenting Training Program, “Limits” 3 efficacy and the use of parenting practices and to what extent this A quasi-experimental method using a controlled before-and-after was perceived by adolescents and facilitators. In light of that, some design (Mertens, 2010) was carried out to evaluate the effects of a improvements and recommendations are suggested to improve the parenting intervention program, “Limits”, for families of juvenile program and its implementation. The article gives evidence on the offenders that has been applied over the past decade in Catalonia, first rigorous evaluation process of the “Limits” program in the frame Spain. A nonequivalent comparison group design with multiple of public administration accountability, where few such evaluations informants and mixed (quantitative and qualitative) methodology have been attempted. As Slavin (2008) and Spiel (2009) indicated, was used. an evidence-based assessment uses research and theory to guide the By comparing the two groups (intervention and comparison evaluation in all its phases. This article provides some preliminary groups), the study seeks to answer the following question: Is there evidence that contributes to the “Limits” program as potentially any statistically significant difference in parents’ perception of self- being an evidence-based program. efficacy and in the use of parenting practice between the two groups before and after the implementation of the intervention? Evaluation Plan and Intervention of “Limits” It is expected to find the “Limits” program helps those families who receive the intervention to improve their perception of self- The “Limits” program (Larriba, 2010) is a second-generation efficacy and the use of their parenting practices in comparison to bidirectional parenting program (Martín Quintana et al., 2009), those families who do not. focused on the quality of parents’ and children’s interactions in daily activities, attachment, empathy, setting limits and managing Sample inappropriate and/or aggressive behaviors. The program targets families with a higher risk of disruptions than other families in the Ninety-one parents being monitored by the juvenile justice general population. The content of the program is based on the social teams in Catalonia were identified by their caseworker as eligible to development approach (Mulford et al., 2016), and is also based on the participate in the “Limits” program according to different criteria. research on parent training and parenting skills (Mulford & Redding, The participating sample was divided into an intervention group 2008) and on the support and control dimensions in parenting (IG) (n = 60) and a waiting list comparison group (CG) (n = 31). All (Hoeve, Dubas, Eichelsheim, van der Laan, Smeenk, & Gerris, 2009). the families who agreed to participate constituted the intervention The “Limits” program is aimed at families with adolescents aged group. Meanwhile, those families who did not agree or were not 14 to 16 who are in the juvenile justice system because they have available to participate in the program edition (conflicts with work committed an offense, whether proven or alleged. It is addressed to schedules, other responsibilities, etc.) made up the comparison group that segment of population identified as being exposed to a particular and were put on the waiting list to attend a following edition of the risk (drug abuse or other behavior problems and risk factors such program. Therefore, participants in the program condition were as poor social adaptation or school dropout). It is a parenting skills compared with their counterparts in reported parenting practices, training program, selective and preventive in nature, which provides before and after the intervention. active training in parenting skills (to improve communication Six parents in the CG were excluded from the analysis as they skills with their adolescents, the parent-adolescent relationship, reported that they had participated in other parenting interventions strengthen family ties, etc.). The two-month program involves 8 previously and a further 5 were excluded because they did not do face-to-face weekly group sessions (lasting two hours each) in the post-test. Six parents in the IG dropped out of the program or which different activities are delivered by facilitators: presenting had attended less than 50% of the sessions and 1 did not do the post- information, conducting exercises, promoting group discussions, test; despite finishing the program, none of them were included in and group dynamic, skill practices, etc. The contents are: learning to the analysis. The final valid sample for the comparisons pre-post test define change of behavioral goals, communication skills, reducing consisted of 73 parents divided into two groups: the intervention conflicts and improvement of family relations, establishing norms group receiving the program (n = 53) and the comparison group and limits, supervision, sanctions and family ties, problem solving, (n = 20). Those 73 parents represented a total 59 families that had family position with regards to tobacco, alcohol and other drugs, participated either in the intervention group (n = 42 families) or were and monitoring. Every session includes the delivery of short written part of the waiting list comparison group (n = 17 families). All parents support material for parents and the assignment and completion participated voluntarily. of various tasks to do at home with the aim of applying positive Adolescents also took part in the study and were interviewed parenting skills and establishing boundaries with adolescents in their (using a single-question survey) to determine their perceptions of day-to-day life. Tasks at home should be practiced during the week, their parents’ behavior after the program; 47 adolescents completed in between sessions. The task assignment is reviewed over the next the survey after they received prior parental consent (35 from the IG program session. and 12 from the CG). The program has a 130-page treatment manual in which each session (120 minutes) is clearly explained and protocolled with Characteristics of the Families a very precise timing for every activity, precise explanations, and methodological recommendations. It also includes written Of the 53 parents that participated on the “Limits” program, information to be delivered to participants. A total of 351 parents 63.5% were mothers, 28.8% were fathers, and 7.7% legal guardians. have participated in different applications of the “Limits” program With regard to the parents’ nationality, 70% were Spanish. The across Catalonia since its conception in 2007. mean age of parents in the IG was 49.13 (SD = 7.95) and their The program facilitators are supervised by the Center for Legal adolescents’ was 17.6 (SD = 1.14). The distribution by gender Studies and Specialized Training (CEJFE), a Catalan government was 75.6% boys and 24.4% girls, with a mean of 1.8 (SD = 1.01) body reporting to the Department of Justice that provide them with siblings. Almost half of the parents in the IG (42%) had at least training and technical assistance. CEJFE compiled a protocol to clarify primary education, while over 60% of their adolescents had not procedures so as to ensure consistency in training and measurement, completed secondary; 70% of the parents reported having a job and also ensured that all ethical requirements were met. and 18.5% said they received welfare benefits; 81.5% reported having enough income to sustain themselves and their family. Method The CG had similar socio-demographic characteristics to the IG 4 M. Martínez-Muñoz et al. / Psychosocial Intervention (2019) 28(1) 1-10 and their adolescents were also involved in the juvenile justice Table 1. Sociodemographic Characteristics and Target Adolescents Profile system. There were no significant differences between IG and CG with regard to demographic data (see Table 1), except for the type Intervention Comparison of family variable: in the IG, the majority consisted of two-parent group group Item n = 53 n = 20 p families (81.3%) while half of the parents in the CG consisted of % % single-parent families (50%), χ²(1, 73) = 21.352, p =< .0005. M (SD) M (SD) Most of the target adolescents in the IG were studying (61.5%), Target parents although 34.6% were neither working nor studying. The target Father 28.8 25.0 ns adolescents in the IG presented several problems associated with Mother 63.5 75.0 dabbling with drugs and/or drug abuse (77.6%), and belonging to Legal guardian 7.7 - (grandparents, etc.) dissocial groups (43.4%). There were no significant differences between IG and CG with regard to the target adolescent’s situation Parent’s nationality (see Table 1), except for the number of offenses: more than half Spanish 70.0 65.0 ns Other nationality 30.0 35.0 of the adolescents (57.7%) in the IG had committed only one Average age of parents (years) 49.13 (7.95) 45.10 (7.26) ns offense, whereas the percentage of adolescents in the CG who had Average age of target adolescent committed either 2 or 3 (40%) or more than 3 (35%) offenses was (years) 17.60 (1.14) 18.00 (1.66) ns higher than in the IG (38.5% and 3.8%, respectively), χ²(1, 73) = Sex of target adolescent 14.906, p =< .005. Male 75.6 89.5 ns Female 24.4 10.5 Fidelity and Program Adaptation Other siblings at home 1.81 (1.01) 1.80 (1.15) ns Type of family A total of 16 professionals selected from the Catalan Justice 2 parent family 81.3 35.0 < .005 Single parent 12.5 50.0 Department’s specialized teams (EMO: Equips de Medi Obert, open Reconstituted 6.2 15.0 environment intervention teams, and SMAT: Serveis de Mediació Parent´s current work situation i Assessorament Tècnic: mediation and technical advice teams) Working 70.0 60.0 ns implemented the program according to agreed standards, leading a Not working, seeking job 15.0 25.0 total of 9 groups in 2013. Not working 15.0 15.0 All the professionals took a training course, provided by the Income to cover basic expenses Catalan Administration, to ensure adherence and consistency in the Yes 82.7 75.0 ns implementation of the program and also the quality of delivery to No 17.3 25.0 make it effective. They were trained in the core components of the program as well as in the procedures on how to implement it. Welfare help First-time caseworkers (facilitators) who implemented the Yes 18.5 35.5 ns program were monitored by another member of the team with No 81.5 64.5 Parents highest level of education previous experience of its implementation to provide corrective Primary education 42.0 65.0 ns feedback and promote positive outcomes (delivering the prescribed Secondary education 6.0 15.0 content and activities, engaging participants in the session, using unfinished appropriate methodology, etc.). Those behavioral observations Secondary education 12.0 10.0 Professional training 20.0 - provided a more objective assessment of the program implementation College 12.0 - and decreased the risk of having large deviations from the original Other 8.0 10.0 content of the program. Adolescents’ highest level of Also, facilitators were asked to complete a checklist after every education session using direct observation in order to: a) compare participants’ Primary education 15.4 10.0 ns changes in program outcomes in each session, b) capture participants’ Secondary education 61.5 75.0 unfinished responsiveness to and attendance at the program, and c) control Secondary education 13.5 10.0 adherence to criteria established in program implementation. Others 9.6 5.0 All professionals involved (those who implemented the program Current adolescent activity and those who only did the referrals) also received a full day’s training Studying 61.5 52.5 ns provided by researchers at the university in order to consistently Not studying and not working 34.6 47.5 Other 3.9 - apply all data sources and protocols to both groups (IG and CG) during program implementation. Number of offenses by adolescent First offense 57.7 25.0 < .005 2 or 3 38.5 40.0 Attendance More than 3 3.8 35.0 Adolescent drug abuse Parents in the IG attended an average of 7.1 (SD = 1.10) of the 8 Flirting 22.4 25.8 ns Abuse 55.2 32.3 planned sessions. Of the 53 parents in the IG, 25 (47%) attended all 8 No consumption 15.5 22.6 sessions, 14 (26.5%) participated in 7 sessions, 9 (17%) participated No information available 6.9 19.3 in 6 and 5 (9.5 %) in 5 or fewer sessions. Participants attending less Belonging to dissocial groups than half of the sessions were excluded from the analysis. When Yes 43.4 54.8 ns parents missed a session, the professional had a meeting with them No 45.3 41.9 No information available 11.3 3.2 to explain the content missed in the previous session. It was also ensured that all ethical requirements were met. All participants Note. ns = non-significant. signed a written consent form and data was delivered anonymously to researchers so as to preserve confidentiality and participants’ anonymity. Measures Evaluation of a Parenting Training Program, “Limits” 5 To measure the results of the intervention, different data sources Observational checklist of parenting practices by caseworkers/ were designed and used to collect data from the perspective of the facilitators. The facilitators used this tool in each session to record different agents involved (parents, adolescents, and caseworkers- the changes observed in outcomes and parents’ use of parental facilitators). practices in their day-to-day lives. Dichotomous responses (e.g., Use of parenting practices and measure of self-efficacy. An “did the person get involved in the session? did the person try to ad hoc questionnaire was created and validated to measure the use apply the strategies that he/she had learned in the session?”), rates of parenting practices and parents’ self-efficacy perception on the (e.g., number of sessions attended), and open ended responses (e.g., practices taught on the “Limits” program. A tailored and versatile “which strategies he/she had applied during the session; reasons for instrument was required in order to gather detailed and specific data not attending the sessions”) were used to systematically collect the from the performance indicators of the program. relevant information concerning the progress (or lack of it) of the The design of the questionnaire was conducted in four phases: participants in the program. All the questions from this tool were a) revision of literature regarding parent’s self-efficacy and parental anchored to the specific behavior or activities from the “Limits” practices; b) design of the instrument based on the performance program. It was validated by two caseworkers who were asked about indicators directly related to the program’s objectives; c) revision of the relevance of the items and the usefulness of the instrument. the instrument, carried out by 7 experts (4 experts in juvenile justice Some changes were made according to their suggestions before its programs and 3 caseworkers), in terms of relevance and clarity of application. scales items and dimensions – in line with this revision some changes were made: items deleted, rephrasing other parts, etc. to increase Data analysis content validity; and d) first application of the questionnaire to a sample of 9 parents (all of them potential participants in the “Limits” A descriptive chi-square analysis for socio-demographic data program) in order to obtain information about the accessibility of the and baseline variables was used. A generalized linear mixed model items and format – this phase generated preliminary information comparing outcomes of the IG and CG was employed. about its internal consistency. After that phase, further improvements “Type of family” and “number of offences” (both statistically were made to the final version of the instrument. Finally, the field different between IG and CG) were included in the model as co- work was conducted, and the questionnaire was applied to the variables to control their possible effects on the final parents’ self- participant families to determine the use of parenting and self- efficacy outcomes. As the analysis included more than one parent efficacy practices of parents and to generate more information with from the same family we took into account the non-independence regard to its psychometric properties (validity and reliability). of observations considering “family register” as a random factor in The final questionnaire included 17 items and was divided the model. into two sections: in the first, parents were requested to indicate A conventional content analysis was conducted using coding on a Likert scale from 0 (not at all) to 4 (a lot) how confident they categories emerging directly from the text data (Hsieh & Shannon, felt about the practices before and after program implementation 2005). Once the researchers agreed upon and consistency was found (internal consistency, alpha = .89). In the second section (on the post- in the coding, data was coded and analyzed using a descriptive test questionnaire), parents were asked to give a Yes or No response analysis matrix. about whether they had used each of the 17 practices over the past two months. An exploratory factorial analysis with varimax rotation was Results conducted in order to give additional evidence on the internal validity of the questionnaire by exploring the underlying structure The results presented here are based on parents’ perceptions and its components. regarding their self-efficacy and use of the parenting training The analysis revealed the presence of a six-factor solution practices in their daily lives, together with those of the professionals explaining 76.7% of the variance with alphas between .56 and .84 and the target adolescents. Before describing the results, some (see Table 2): 1) communication skills (e.g., “I feel I have the skills relevant features of the sample are described. to tell my son/daughter how to behave”); 2) family perception of tobacco and other drugs (e.g., “I feel I have the skills to tell my son/ Effects of Intervention on Parents’ Self-efficacy and Use of daughter my opinion about drugs, tobacco and alcohol”); 3) positive Parenting Practices reinforcement and family relationships (e.g., “I feel I have the skills to find new recreational spaces to do activities with my son/daughter”); Pre-test comparisons between the IG and CG showed no 4) management of punishments (e.g., “I feel I have the skills to decide differences in the measure of parenting self-efficacy within the 6 which punishment to give to my son/daughter”); 5) rules and limits subscales (program outcomes), so the two groups do not initially (e.g., “I feel I have the skills to decide which limits to place on my differ in the dependent variable. A generalized linear mixed model son/daughter’s behavior”); and 6) emotional self-control (e.g., “I feel I was performed (Table 2) and the results indicated no significant have the skills to control my reactions when I get angry with my son/ difference either between the two groups after the intervention on daughter”). the overall self-efficacy measure and within the 6 subscales, F(1, 73) Therefore, the results from the psychometric analysis showed = 0.305, p = .582. that the questionnaire has great potential to determine the use of However, a time-effect analysis indicated that the IG improved parenting practices and parents’ self-efficacy in the context of the significantly in terms of self-confidence in 2 of the 6 parental “Limits” program. Ongoing efforts should provide more conclusive practices performed (gray-colored cells, Table 2). Parents in the IG evidence through a confirmatory factorial analysis. reported feeling more confident in “giving positive reinforcement and A single question survey to the adolescents. A single item was family relationships” and “emotional self-control” as a result of their used to triangulate adolescents’ and parents’ perceptions regarding participation in the program. The CG also improved significantly in the application of parental practices at home (e.g., “In the last two these two subscales during the pre- and post-test interval. Therefore, months, have you noticed a change in your mother and father’s no treatment effects between both groups were observed on parents’ behavior towards you? If so, what changes have you noticed?”). It was self-efficacy perception. validated by a group of three caseworkers who were asked about the Regarding the use of parental practices, over 70% of the parents relevance and appropriate formulation of the question. in the IG reported applying all the parental practices taught in the 6 M. Martínez-Muñoz et al. / Psychosocial Intervention (2019) 28(1) 1-10 Table 2. Results of Group by Time Type of Num. of Pre-test Post-test Group (A) Time (B) AxB family offences Questionnaire scales IG CG IG CG n = 53 n = 20 n = 53 n = 20 F F F F F M (SD) M (SD) d M (SD) M (SD) d Overall self- efficacy measure 3.08 2.80 3.20 2.83 for parenting 0.28 0.37 2.43 0.98 0.30 0.60 2.43 (0.17) (0.19) (0.18) (0.18) practices (α = .89) Factor 1: communication 3.25 3.05 3.34 2.92 0.20 0.42 1.6 0.02 1.06 0.62 2.92 skills (0.21) (0.23) (0.20) (0.20) (α = .84) Factor 2: family perception 3.67 3.49 3.64 3.21 of tobacco and 0.18 0.43 1.96 2.12 1.43 1.05 2.55 (0.19) (0.21) (0.18) (0.19) other drugs (α = .81) Factor 3: positive reinforcement 2.80 2.42 3.09 2.73 0.38 0.37 1.61 6.81* 0.001 0.57 1.30 and family (0.25) (0.27) (0.25) (0.26) relationships (α = .74) Factor 4: emotional self- 2.76 2.16 3.12 2.88 0.60* 0.24 3.30 20.70** 2.26 0.30 0.42 control (0.20) (0.22) (0.20) (0.21) (α = .83) Factor 5: management of 2.59 2.54 2.78 2.44 0.06 0.34 0.33 0.12 1.03 0.44 1.51 punishments (0.30) (0.31) (0.29) (0.31) (α = .79) Factor 6: 3.16 2.95 3.08 2.90 rules and limits 0.21 0.18 0.58 0.20 0.007 0.68 2.53 (0.23) (0.26) (0.22) (0.23) (α = .56) Note. IG = intervention group; CG = comparison group. *p < .05, **p < .005. program in real life (program outcomes). Parents in the CG performed favor of the IG, χ²(1, 73) = 4.399, p < .05 (see Table 3). similarly. However, there were significant differences in the parental However, the strategies least applied in the CG (70%) are those practice “communicating limits of behavior” between IG and CG in related to deciding, communicating, and enforcing a sanction using Table 3. Use of Parental Practices in Day-to-day Life according to Parents’ Perception Intervention Comparison Parental practices applied during the pre/post-test interval group group p % % Talk to my child about how he/she must behave 98.00 100.00 ns Try to improve communication with my son/daughter 96.20 95.00 ns Talk about important issues with my son/daughter 94.20 100.00 ns Talk with my son/daughter every day 90.20 100.00 ns Control my reaction to my son/daughter when I get angry 94.00 89.50 ns Tell my son/daughter positive things about him/her 90.00 85.00 ns Decide the limits I place on my son/daughter’s behavior 87.50 85.00 ns Communicate to my son/daughter the limits on their behavior 95.80 80.00 < .05 Ask about and find out more things about my son/daughter (tastes, what they do in their free time, etc.) 94.11 85.00 ns Tell my son/daughter the positive things he/she does 96.10 85.00 ns Find new recreational spaces to do activities with my son/daughter 71.40 52.50 ns Decide what sanctions and punishments to apply to my son/daughter 84.00 70.00 ns Communicate the sanctions and punishments 81.60 73.70 ns Enforce the sanctions and punishments 81.30 70.00 ns Solve problems with my daughter/son in a relaxed way 95.80 90.00 ns Talk with my son/daughter about drugs, alcohol and tobacco 90.00 95.00 ns Express what I think about drugs, alcohol and tobacco to my daughter/son 92.00 95.00 ns Note. ns = non-significant. Evaluation of a Parenting Training Program, “Limits” 7 the content strictly related to the program, while over 80% of the (case P-007-01-M-I). parents in the IG reported applying them. In general, around 65% of professionals reported that participants displayed a good level of involvement in each session. Adolescents’ Perceptions of their Parents’ Parenting Abilities Regarding the parental practices used during the pre/post-test interval, the professionals reported that the strategies implemented Forty-seven adolescents responded to the survey (35 from the IG by most participants are related to communication skills and and 12 from the CG); 65.7% of the adolescents from families in the IG improving family relationships (66% and 64.2% of the participants noticed positive changes in their parents’ behavior in the previous applying these strategies, respectively). It seems that these skills two months as a result of their participation in the “Limits” program. were the ones attempted most during the pre/post-test interval from Most parents were reported to have been more communicative and the professionals’ perspective, probably because communication is a to have exercised greater self-control after the program: basic skill found in any kind of interpersonal interaction. To a lesser “She is calmer now, and she doesn’t get so angry, she talks to me extent with other strategies implemented by families were those more.” (case A-047-05-M-I). related to setting norms, rules, and limits (62.3%). Strategies related Some of the adolescents recalled how their parents had tried to to family problem solving and family position regarding tobacco, establish a closer relationship with them and reinforce them in a alcohol, and other drugs were applied, according to a professional’s positive way: points of view, only by a little more than half of the participants. “She gives me more support; our communication has improved.” The difficulties and facilitating factors were reported by the (case A-072-02-M-I). facilitators. The characteristics of the person participating in the Some adolescents also noticed that they had had fewer arguments program, including their attitudes, emotions, and skills, difficulties in the previous two months and that the communication and their seeing the problem (due to excessive focus or lack of perception relationship with their parents had improved in general: of the problem), and discrepancies and difficulties with their own “There is more communication. She has improved in everything; partner made up the bulk of the problems perceived by professionals. there are no arguments and fights like the ones we had before.” (case Some characteristics of the adolescents, such as their personality, A-013-02-F-I ). impulsiveness, reaction to interactions, substance abuse, or Some also reported that their parents had been more able to persistence in challenging behaviors, were also hindering elements. establish limits and rules in a more assertive way. Finally, the lack of a relationship, and the communication difficulties, Out of the adolescents from families in the CG, 58.3% also noted were other highlighted elements. changes in their parents’ behavior despite their not participating “Lack of communication with the partner and discrepancies over in the program. The reported changes were related to improving criteria.” (case P-073-02-M-I). communication and greater emotional self-control. However, after “The high intake of drugs and the refusal to reach an agreement.” applying a chi-square analysis, no significant differences were found (case P-044-05-P-I). between the CG and IG with regard to adolescents’ perceptions. “The lack of a more intensive relationship.” (case P-047-05-M-I). Regarding facilitating factors, professionals emphasized attitudes or attitude change in parents: belief in the possibility of change, Professionals’ Perceptions of Parents’ Parenting Abilities awareness of change in oneself and in the adolescent, adolescent’s vision from a new perspective and not only casting blame. The Some difficulties and aptitudes in applying the parental practices mobilization of parents’ personal characteristics (persistence, were also observed and reported by facilitators. So, facilitators made perseverance, etc.), awareness of own mistakes, acknowledging some adaptations to some of the components of the program in the adolescents’ positive features, increasing communication, and order to meet participants’ needs and profile better. The information sharing space and time all became enhancing elements of a new provided by professionals was useful for contrasting adolescents stage in which both agents recognize and acknowledge each other’s and parents’ feelings of self-efficacy and perceptions of their use of parental practices at home, and for capturing participants’ actions and efforts. engagement to the “Limits” program. “Acknowledging the positive changes in the adolescent. Following the implementation of the program, facilitators were asked Cooperation within the couple to overcome problems.” (case P-005- about parents’ level of involvement in each session and their use of 01-M-I). parental practices taught in each session during the pre/post-test interval. “Belief in the possibility of change for his daughter and himself.” Sixty-eight percent of the professionals reported that parents (case P-018-02-P-I) increased their overall awareness and capacity to address their adolescents’ behavioral problems after the program. By contrast, Discussion other professionals had reservations about parents’ improvement, in some cases reporting that they observed a change in parents’ behavior Program evaluation is complex and many programs do not meet but had doubts about the true internalization of practices and noting the minimum requirements of evaluation. In the case presented here, that more time would be necessary to determine the effects of the in order to overcome some methodological limitations described in program. In still other cases, professionals mentioned that change previous reviews (Letarte et al., 2010; Rodrigo et al., 2009; Sanders et was not possible due to the existence of other serious circumstances al., 2007), a set of strategies were included in designing the program regarding the adolescents’ behavior. evaluation: a comparison group was used and samples of 30 parents “He has generated new attitudes and ways of communication or more were expected to be included in CG and IG. Also, the point of with the adolescent, although these were positive from the outset.” view of both facilitators and target adolescents were included in the (case P-072-02-M-I). design as a way of improving the consistency of the results. “He lacks the time and support to internalize the program and use The evaluation of the “Limits” program revealed no statistically it.” (case P-020-02-M-I). significant differences between IG and CG in the pre-test and “We think he needs a continuation to consolidate all the small post-test measures of parents’ self-efficacy on any subscale. These changes and the awareness he has gained with the group. He shows results are in accordance with those of other evaluations of similar an interest in putting everything worked on in the sessions into programs addressed at improving parental practices (Letarte et al., practice, but his son’s problems are bigger than his ability to act.” 2010). Regarding emotional self-control, both groups showed an 8 M. Martínez-Muñoz et al. / Psychosocial Intervention (2019) 28(1) 1-10 improvement after post-test. The fact that both groups used juvenile To sum up, the program helps raise awareness of problems related justice services (basically advising and counseling through interview to family relationships between parents and adolescents and the sessions with the Services of Mediation and Technical Advice - SMAT) need to establish rules and boundaries to promote cohabitation in a may have had a collateral effect on results. framework of respect and effective communication. This awareness The implementation of communication strategies was significantly can also be related to the results obtained; the program and its higher among the IG than the CG. Strategies related to communication evaluation tools generate an effect of awareness that could influence were reported as the most widely used in this study. In this respect, post-test responses as the result of a more critical and sensitive vision the program appears to have had a clear impact on the most basic of the situations arising in this area. Professionals stated that families aspects of relations between parents and adolescents, showing were more critical of their respective situations after participating in significant differences from the CG. Communication strategies do the program. This effect is attributable to their having answered a improve the quality of the relationship with adolescents but do not questionnaire and has been identified in other spheres, contributing necessarily affect the application of other skills such as setting rules to the idea that people answering a questionnaire become sensitized and sanctions. As communication is the vehicle for the majority of as an effect of the response process itself (Lajunen & Summala, 2003). interactions between parents and adolescents, communication Along those lines, the program seems to have a positive impact on strategies are probably the first resource to be mobilized as they improving skills related to communication and improving parents- can be used in the practice of other parenting skills. The program adolescents relationship dynamics (Mulford et al., 2016; Taylor et al., therefore seems to contribute to the initial implementation of simple 2015) but fails to achieve its objectives in relation to establishing rules strategies that can then provide the basis for deploying other skills. and sanctions. Thus, the evidence seems to suggest that “Limits” is This result is consistent with what Mulford et al. (2016) suggested in ineffective at changing its primary target behavior. It would seem, as their meta-analysis. mentioned, that more time is required for interaction and monitoring In accordance with previous studies, it was found that a more actions should be incorporated after the program has finished. Finally, prolonged intervention is needed for the program to take effect, and in accordance with that suggested by Chacon, Sanduvete, Portell, particularly when there are difficulties between parents and and Anguera (2013), it would be important to adapt the participant adolescents (Letarte et al., 2010; Mulford et al., 2016; Webster et al., selection procedure to ensure that participants’ profile matches the 2012). The sample presented added difficulties associated with the aims and features of the program and guarantee its success; findings characteristics of target adolescents: they were older adolescents, show that the program was applied to families with adolescents aged flirting with drugs or drug abuse, belonging to dissocial groups and 18 on average who had already been exposed to a particular risk, having committed more than one offense. These traits could have instead of the age range of 14-16 (its original target). an impact on relationships among family members and the lack of There is still a lot of work to be done regarding both the program communication. itself and its evaluation. Nevertheless, this work is a first attempt As a strategy to triangulate information, the perceptions of the to draw attention to the need of delving into the evidence based parents, their adolescents and professionals concerning parents’ approach to both the practice (program design and implementation) behavior after participating in the program were considered. and its assessment as Axford, Elliott, and Little (2012) suggest in Adolescents’ perceptions were in accordance with IG parents’ self- relation to the promotion of “evidence-based” programs in children’s efficacy results reported in the questionnaire with regard to “positive services, or more recently Vázquez, Molina, Ramos, and Artazcoz reinforcement” and “self-control”. These two scales showed a (2017). The program has not achieved its goals, but this evaluation significant improvement in the IG as a result of the program. For their has identified the way ahead. part, professionals also perceived an improvement in the IG regarding the acquisition of parenting skills for improving relationships dynamics at home, even though parents’ perceptions tended to be Limitations/Future Research more positive than professionals’ in the use of their parental practices in their day to day lives. Both adolescents and professionals agreed that Some limitations emerged from this exploratory evaluation of parents had tried to apply the practices and had improved in them, “Limits”. These limitations need to be considered for a definitive especially those related to increasing the quality of communication. evaluative study of the program and further research using Professionals emphasized their impression that the program had longitudinal designs should be done in order to prove its effectiveness. allowed parents to become aware of their adolescents’ situation and The use of a homegrown outcome questionnaire rather than their relationship and tried to apply communication strategies in a standardized one to measure parents’ self-efficacy could be order to improve the latter. However, their view was more critical in considered a study limitation. However, one of the reasons to use terms of the true internalization of these skills and especially their an ad hoc questionnaire was because it is more versatile to obtain application in daily home life. the needed information and adapted to the contents of the “Limits”. However, it is worth noting that the program is aimed at families Thus, the questionnaire used suggested reasonable construct validity with adolescents aged between 14 and 16 and is currently being and reliability for parental self-efficacy subscales and the evidence applied to families with adolescents aged 18 on average. This would provided corresponded with the measures used in other studies on suggest the need to design and deploy more specialized programs – parental behavior. where prevention would be secondary or tertiary – aimed at the more Even though some strategies were used to ensure fidelity (initial complex specific problems facing families. If the target adolescent is training monitoring, etc.) more methods and data sources to measure 18, other issues arise and the program no longer plays a preventive fidelity should be used in future research to achieve more accurate role. In fact, the older the boy or girl, the less effective these kinds and possible stronger results. of programs (Webster et al., 2012). In his regard, it seems clear, and The groups were not assigned randomly to conditions, but using a the results are consistent with this idea, that early intervention is key quasi-experimental controlled before-and-after study. Nevertheless, when the program is aimed at addressing risk factors that become it would be desirable to use higher levels of randomization to problems over time. There may be limitations in the possibilities of minimize the risk of contamination from other variables that could developing such programs in the institutional environment of juvenile bias the intervention effects. However, the initial difference between justice in the near future and cases will continue to be diverted to the groups (IG and CG) in different socio-demographic data and baseline “Limits” program because it is one of the few specific programs that variables were considered in the analysis to discern better if the exist and is available to open prison and juvenile justice professionals. differences observed between both groups were attributed to the Evaluation of a Parenting Training Program, “Limits” 9 intervention and not to this existing profile differences. The initial professionals, and researchers to approach the “Limits” program to the results showed no significant differences between the IG and CG as indicators of evidence-based programs (Rodrigo, 2016). they were having equivalent scores in the pretest. Most probably, a longitudinal evaluation design would have identified the true impact Conflict of Interest of “Limits” in participants. Facilitators were not blind to condition, and that could have an The authors of this article declare no conflict of interest. effect on their data on parent success. For similar future research, it could be necessary to consider observational data instead of Notes facilitator perceptions. Another limitation is associated with the nature of measurement 1 This study reports findings from an independent evaluation (pre/post-test questionnaires). The measures were self-reported of the Limits program implemented within the juvenile justice and the influence of socially desirable response tendencies should services in Catalonia. The authors have no affiliation with the therefore also be considered. That said, the participants were asked developers or distributors of the “Limits” program. to respond sincerely and honestly, so as to minimize inaccurate responses. It would be recommendable to use interviews with the parents to contrast questionnaire response regarding the use/transfer References of parenting practices at home or other observational measures. Amoros, P., Balsells, M. A., Mateos, A., Jose, R. M., & Vaquero, E. (2014). As Hamilton and Carr (2016) stated, self-reported scales are “Aprender juntos, crecer in familia” program for the development one of the main types of instrument for assessing family. In the of resilience and positive parenting. Second World Congress on context of the “Limits” program, evaluating the transfer of learned Resilience: From person to society (pp. 411-414). Timisoara, Romania. Axberg, U., & Broeberg, A.G. (2012). Evaluation of ‘‘The Incredible Years’’ in skills through self-reported procedures also raised methodological Sweden: The transferability of an American parent-training program problems, as it is very difficult to know whether the skills learned to Sweden. Scandinavian Journal of Psychology, 53, 224-232. https:// in the course were being practiced effectively at home. Therefore, doi.org/10.1111/j.1467-9450.2012.00955.x Axford, N., Elliott, D. S., & Little, M. (2012). Blueprints for Europe: Promoting there is a double inference: the first arising from the perception of evidence-based programmes in children’s services. Psychosocial families about their reality and practice, and the second derived Intervention, 21, 205-214. https://doi.org/10.5093/in2012a11 from professionals’ reports about what families claim. This double Bartau, I., & De la Caba Collado, M. A. (2009). Una experiencia de mejora de las habilidades para la parentalidad y el desarrollo sociopersonal inference, added to the percentage of random error (which may be de los menores en contextos de desprotección social. Intervención present in a population of these characteristics), suggests caution Psicosocial, 18, 131-151. https://doi.org/10.5093/in2009v18n2a5 should be applied in interpreting the data. Buzhardt J., Rusinko, L. Heitzman-Powell, L., Trevino-Maack, S., & McGrath, A. (2015). Exploratory evaluation and initial adaptation of a parent training programme for Hispanic families of children with autism. Implications and Future Directions Family Process, 55, 107-122. https://doi.org/10.1111/famp.12146 Chacon, S., Sanduvete, S., Portell, M., & Anguera, M.T. (2013). Reporting a program evaluation: Needs, program plan, intervention, and decisions. According to the evidence presented, some modifications need to International Journal of Clinical and Health Psychology, 13, 58-66. be made prior to future implementation or further expansion of the https://doi.org/10.1016/S1697-2600(13)70008-5 program in order to improve its usability and effectiveness with its Council of Europe. (2006). Resolution 19 of the Committee of Ministers to member states on policy to support positive parenting. Adopted by the target population. Committee of Ministers on 13 December 2006 at the 983rd meeting of It seems necessary to maintain the same initial training for the Ministers’ Deputies. practitioners to ensure program consistency. It would also be Domenech Rodriguez, M., Baumann, A., & Schwartz, A. (2011). Cultural adaptation of an empirically supported intervention: From theory to desirable to monitor the use of parental practices and parent self- practice in a Latino/a community context. American Journal of Community perception of those practices over time or 3 or 6 months after Psychology, 47, 170-186. https://doi.org/10.1007/s10464-010-9371-4 program implementation to verify its durability and provide feedback Fernandez, M. A., & Eyberg, S. M. (2009). Predicting treatment and follow- up attrition in parent–child interaction therapy. Journal of Abnormal on the program itself. This follow-up evaluation would also provide Child Psychology, 37, 431-441. https://doi.org/10.1007/s10802-008- evidence on which to base the design and implementation of a set of 9281-1 follow-up programs. Fletcher, R. Freeman, W., & Mathey, S. (2011). The impact of behavioural parent training on fathers’ parenting: A meta-analysis of the triple Despite the fact that “Limits” was designed as a preventive p-positive parenting program. Fathering, 9, 291-312. https://doi. program addressed at families with some incipient communication org/10.3149/fth.0903.291 and relational difficulties and it is necessary to maintain it as such, Forgatch, M. S., & Patterson, G. R. (2010). Parent Management Training - Oregon Model: An intervention for antisocial behavior in children the objectives of the program need to be readjusted to make them and adolescents. In J. R. Weisz & A. E. Kazdin (Eds.), Evidence-based achievable within 8 sessions. Other authors have already stressed psychotherapies for children and adolescents (pp. 159-177). New York, the need to extend the intervention period to find evidence of NY: Guilford Press. Gilmore, L. A., & Cuskelly, M. (2008). Factor structure of the parenting sense effectiveness and impact. Some other improvements should also of competence scale using a normative sample. Child Care, Health & be considered in order to achieve a better cultural and linguistic Development, 38, 48-55. adaptation to participants’ profile and their context, as suggested Hamilton, E., & Carr, A. (2016). Systematic review of self-report family assessment measures. Family Process, 55, 16-33. https://doi.org/10.1111/ for other parental programs addressing specific populations with famp.12200 specific needs (Buzhardt, Rusinko, Heitzman-Powell, Trevino-Maack, Hoeve, M., Dubas, J. S., Eichelsheim, V. I., Van Der Laan, P. H., Smeenk, & McGrath, 2015; Domenech Rodriguez, Baumann, & Schwartz, 2011). W., & Gerris, J. R. (2009). The relationship between parenting and delinquency: A meta-analysis. Journal of Abnormal Child Psychology, The results of this study have demonstrated the need to design 37, 749-775. https://doi.org/10.1007/s10802-009-9310-8 complementary tailored and remedial programs aimed at solving Hsieh, H. F. & Shannon, S. E. (2005) Three approaches to qualitative content serious and specific problems already detected in the target group analysis. Qualitative Health Research, 15, 1277-1288. https://doi. (i.e., drug abuse, child-parental violence), for which “Limits” could org/10.1177/1049732305276687 Lajunen, T., & Summala, H. (2003). Can we trust self-reports of driving? not address effectively as it is conceived as a preventive program. Effects of impression management on driver behaviour questionnaire Finally, it would also be desirable to improve the evaluation system responses. Transportation Research, F6, 97-107. https://doi.org/10.1016/ of the program itself and to include it as a core part of the program. S1369-8478(03)00008-1 Larriba, J. (2010). Límits: programa d’actuació preventiva familiar de la Last but not least, it would also be highly advisable to promote the transgressió i del consum de drogues en menors i joves que passen pel partnership scheme bringing together policy makers, service providers, circuit de la justícia Juvenil. Generalitat de Catalunya. Departament de 10 M. Martínez-Muñoz et al. / Psychosocial Intervention (2019) 28(1) 1-10 Justícia. Barcelona, España. competencias parentales en contextos de riesgo psicosocial. Letarte, M. J., Normandeau, S., & Allard, J. (2010). Effectiveness of Intervención Psicosocial, 18, 113-120. https://doi.org/10.5093/ parent training program. Incredible years in a child protection in2009v18n2a3 service. Child Abuse & Neglect, 34, 253-261. https://doi.org/10.1016/j. Salari, R., Ralph, A., & Sanders, M. R. (2014). An efficacy trial: Positive parenting chiabu.2009.06.003 program for parents of teenagers. Behaviour Change, 31, 34-52. https:// Lundahl, B. W. Nimer, J., & Parsons, B. (2006). Preventing child abuse: A doi.org/10.1017/bec.2013.31 meta–analysis of parent training programs. Research on Social Work Sanders, M. R. (2012). Development, evaluation, and multinational Practice, 16, 251-262. https://doi.org/10.1177/1049731505284391 dissemination of the Triple P-Positive Parenting Program. Annual Martín, J. C., Máiquez, M. L., Rodrigo, M. J., Correa, A. D., & Rodríguez, Review of Clinical Psychology, 8, 345-379. https://doi.org/10.1146/ G. (2004). Evaluación del programa “Apoyo personal y familiar” annurev-clinpsy-032511-143104 para madres y padres en situación de riesgo psicosocial. Infancia y Sanders, M. R., Kirby, J. N, Tellegen, C. L, & Day, J. J. (2014). The Triple Aprendizaje, 27, 437-445. https://doi.org/10.1174/0210370042396887 P-Positive Parenting Program: A systematic review and meta-analysis Martin-Quintana, J. C. Máiquez, L. Rodrigo, M. J. Byme, S. Rodríguez-Ruiz, of a multi-level system of parenting support. Clinical Psychology B., & Rodríguez Suárez, G. (2009). Programas de educación parental. Review, 34, 337-357. https://doi.org/10.1016/j.cpr.2014.04.003 Parenting Programs. Psychosocial Intervention, 18, 121-133. https:// Sanders, M. R., Markie-Dadds, C., Rinaldis, M., Firman, D., & Baig, N. (2007). doi.org/10.5093/in2009v18n2a4 Using household survey data to inform policy decisions regarding Martínez González, R. A. (2009). Programa “Guía para el desarrollo de the delivery of evidence based parenting interventions. Child: Care, competencias emocionales, educativas y parentales”. Ministerio de Health and Development, 33, 768-783. https://doi.org/10.1111/j.1365- Sanidad y Política Social. Madrid, España. 2214.2006.00725.x Martínez González, R. A., & Becedóniz Vázquez, C. M. (2009). Orientación Simoes, I., Fonseca, M. F., & Anglin, J. P. (2014). Webster-Stratton Incredible educativa para la vida familiar como medida de apoyo para el Years Basic Parenting Programme (IY) in child care placements: desempeño de la parentalidad positiva. Intervención Psicosocial, 18, Residential staff carers’ satisfaction results. Child & Family Social 97-112. https://doi.org/10.5093/in2009v18n2a2 Work, 21, 198-208. Martínez-González, R. A., Rodríguez-Ruiz, B., Álvarez-Blanco, L., & Slavin, R. E. (2008). Perspectives on evidence-based research in Becedóniz-Vázquez, C. (2016). Evidence in promoting positive education - What works? Issues in synthesizing educational parenting through the Program-Guide to Develop Emotional program evaluations. Educational Researcher, 37, 5-14. https://doi. Competences. Psychosocial Intervention, 25, 111-117. https://doi. org/10.3102/0013189X08314117 org/10.1016/j.psi.2016.04.001 Spiel, C. (2009). Evidence-based practice: A challenge for European McConnell, D., Breitkreuz, R., & Savage, A. (2012). Independent evaluation developmental psychology. European Journal of Developmental of the Triple P Positive Parenting Program in family support service Psychology, 6, 11-33. https://doi.org/10.1080/17405620802485888 settings. Child & Family Social Work, 17, 43-54. https://doi.org/10.1111/ Taylor, L. C., Leary, K. A., Boyle, A. E., Bigelow, K. E., Henry, T., & DeRosier, j.1365-2206.2011.00771.x M. (2015). Parent training and adolescent social functioning: A brief Mertens, D.M. (2010). Research and evaluation in education and psychology. report. Journal of Child and Family Studies, 24, 3030-3037. https://doi. Thousand Oaks, CA: SAGE Publications. org/10.1007/s10826-014-0106-2 Mihalic, S., Fagan, A., Irwin, K., Ballard, D., & Elliott, D. (2004). Blueprints for Vázquez, N., Molina, M. C., Ramos, P., & Artazcoz, L. (2017). Effectiveness of violence prevention. Center for the Study and Prevention of Violence. a parent-training program in Spain: Reducing the Southern European University of Colorado-Boulder. Colorado. evaluation gap. Gaceta Sanitaria. Mulford, C. F., & Redding, R. E. (2008). Training the parents of juvenile Webster-Stratton, C. (1998). Wally’s detective manual for solving problems offenders: State of the art and recommendations for service delivery. at home. (Incredible Years Parents, Teachers, and Children Training Journal of Child and Family Studies, 17, 629-648. https://doi.org/10.1007/ Series). Seattle, WA: Seth Enterprises. s10826-007-9178-6 Webster C., Gaspar, M.F., & Seabra-Santos, M.J. (2012). Incredible Years® Mulford, C., Redding, R. E., & Mendoza, M. M. (2016). Training for effective Parent, Teachers and Children’s Series: Transportability to Portugal parenting. In R. J. R. Levesque (Ed.), Encyclopedia of adolescence. of early intervention programs for preventing conduct problems Cham, Switzerland: Springer International Publishing. and promoting social and emotional competence. Psychosocial Rodrigo, M. J. (2016). Quality of implementation in evidence-based Intervention, 21, 157-169. https://doi.org/10.5093/in2012a15 positive parenting programs in Spain: Introduction to the special issue. Psychosocial Intervention, 25, 63-68. https://doi.org/10.1016/j. psi.2016.02.004 Rodrigo, M. J., Martín, J. C., Cabrera, E., & Máiquez, M. L. (2009). Las