ARTICLE https://doi.org/10.1057/s41599-020-00587-8 OPEN World-wide barriers and enablers to achieving evidence-informed practice in education: what can be learnt from Spain, England, the United States, and Germany? Joel R. Malin et al.# 1234567890():,; A global push exists to bolster the connections between research and practice in education. However, fostering evidence-informed practice (EIP) has proven challenging. Indeed, this ‘pro- blem’ requires simultaneously attending to multiple aspects/levels of education systems, and to the contexts within which they reside. As such, comparative analyses using systems approaches hold potential for achieving context-specific insights regarding how to foster EIP. However, such analyses have been scarce, and what research does exist has generally been limited relative to methods and theory. Given this, the present study executes and describes/reflects upon a novel approach for analysing and comparing EIP in/across systems. In this study, educators’ evidence use patterns are described and comparatively analysed, using a sample of four regions within high-income national settings: Catalonia (Spain), England (UK), Massachusetts (USA), and Rheinland-Pfalz (Germany). This study employs a dual analytical frame (a cohesion/regulation matrix and institutional theory) to supply a methodological lens through which to understand EIP within and across these four systems. Together, this approach not only provides a way of accounting for the macro-level differences between contexts, it also enables a comparison of meso-level and micro-level factors (via institutional theory) that might be common and distinct across systems. This study’s findings reveal substantial diversity in the extent and nature of evidence use between systems, which in turn patterned according to distinctive cultural, sys- temic, and institutional features. Considering these findings, this study’s discussion advances some provisional insights and reflections regarding actual and potential EIP in education. For example, variability relative to the types/extents of accountability pressures, and how this affected educators’ data and evidence use, enabled a discussion holding relevance for policy- makers. We also share process-related insights—i.e., describing the advances and challenges we experienced while undertaking this new approach. These points hold relevance for colleagues wishing to emulate and improve upon the efforts described herein, which we argue are applicable both in and beyond the education sector. Relative to education, these approaches can be applied and improved with an eye toward developing context-specific (vs. one-size-fits-all) packages for fostering EIP and, ultimately, achieving high quality and progressively improving schools/systems. # A list of authors and their affiliations appears at the end of the paper. HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 1 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 T Introduction and aims his paper examines the question of how to bring about both in terms of a given cohort or group of students, or in relation more consistent, evidence-informed practice (EIP) in to an innovation that is required at the level of the school or education globally. To do so, we use a social regulation/ across schools; (2) evidence can aid teachers in the design of new cohesion matrix and an institutional analytic lens to engage in a bespoke strategies for teaching and learning in order to tackle comparative analysis across four contexts: Catalonia (Spain), specific identified problems; (3) evidence can provide teachers England (UK), Massachusetts (MA, United States), and with ideas for how to improve aspects of their day to day practice Rheinland-Pfalz (RP) (Germany). The aim of the analysis is by drawing on approaches that research has shown to be effective; twofold: First, we aim to explore and critique a new way to (4) ideas from research can help teachers expand, clarify and analyse EIP in (and potentially beyond) education. Second, we deepen their own concepts, including the concepts they use to aim to generate insights into how to more routinely foster EIP, understand students, curriculum and teaching practice; and (5) and to ascertain whether there were generalisable lessons from programme evaluations can also provide teachers with specific education that can be applied to other social policy areas. programmes or guidelines, shown by research to be effective, Although not without limitations, a number of insights do emerge which set out how to engage in various aspects of teaching or from our work. These include that research evidence is but one of specific approaches to improve learning. Finally, data can be used the potential influences on practice; furthermore, that use is to assess the impact of embarking on (2)–(5) above. Thus, if contingent on a host of favourable features and conditions, set teachers are able to engage with evidence (both research and data) across micro, meso, and macro levels. Perhaps most significantly, in a way that enables them to undertake any of these actions, their this study demonstrates a new theoretical/methodological teaching quality should be improved. Correspondingly, improved approach for studying EIP in/across systems, which we suggest teaching quality should then lead to improved student outcomes. can be taken up in and beyond the education sector. Although there is now a recognition that evidence use can and should be used to improve practice, there is only limited evidence on how this might be facilitated at the school level (Graves and Evidence-informed teaching Moore, 2017). What’s more, a systemic gap appears to exist Across many countries, national, federal and district level gov- between research and practitioners which as yet shows little ernments are increasingly pursuing approaches to school indication of narrowing (Coldwell et al., 2017; Graves and Moore, improvement that seek to achieve so-called ‘bottom-up’ change. 2017; Whitty and Wisby, 2017). As a result, this leaves only That is, improvements to teaching and learning that are generated sporadic instances of EIP occurring within and across schools by teachers and subsequently shared horizontally and vertically with other factors, such as intuition and experience, instead solely within educational systems. In particular, ‘self-improvement’ is driving much of the decision-making undertaken by teachers now viewed by many as the preferred approach to enhancing (Vanlommel et al., 2017). The danger then is that misconception, educational provision at the school and system level (Greany, biases, and fallible ‘fast’ decisions (Kahneman, 2011) are as likely 2015). An approach often chosen to support self-improvement is to influence teachers’ decision making as much as high quality that of EIP. EIP involves fostering situations in which teaching evidence that helps the identification of problems or that point to practice is consciously informed by evidence derived from: (1) effective solutions. A key question therefore is what can we do to formal research produced by researchers; (2) practitioner enquiry; achieve EIP? In other words, how can we can get school princi- and/or (3) routinely collected school or system-level data (for pals and teachers within education systems, globally, to system- example, student assessment data). A focus on EIP is not without atically use high quality academic research and other forms of merit and there is a nascent but growing evidence base to suggest evidence to improve how they lead and teach? that when teachers engage with evidence, this can lead to In this paper we attempt to shed light on this issue by exam- improvements in outcomes for both teachers and students. For ining EIP, in a new way, in/across four school systems: Catalonia, instance, correlational data indicates that where research is used England, Massachusetts, and Rheinland-Pfalz (RP). (Our selec- as part of high-quality initial teacher education and ongoing tion of these four cases is described further in a subsequent sec- professional development, there is an association with higher tion.) In doing so, we aim both to generate provisional insights school and school system performance (Mincu, 2014). More related to fostering more/better EIP in education, and to set a recently Rose et al. (2017), using a randomised control trial, discussion regarding the merits and drawbacks of the methodo- showed that increased collaborative research use by primary logical and theoretical approaches that we undertook. In addition, school teachers had a positive impact on primary school students’ in light of the aims of this special edition, this paper also seeks to exam results. A range of positive teacher outcomes that emerge ascertain whether there are generalisable lessons from education from collaborative research-informed practice include: improve- that can be applied to improve EIP (and/or its study) in other key ments in pedagogic knowledge and skills, greater teacher con- areas such as health, justice and social care. fidence, and high teacher job satisfaction (Bell et al., 2010; Comparative analyses of evidence use in education have Godfrey, 2016). Similarly, teachers’ use of educational data (e.g. been scarce. Typically, evidence use or research engagement standardised test scores, data used for formative assessment, has not been a primary focus of educational researchers who student self-assessment data, or other data such as attendance) have undertaken comparative study, though these topics still can, in the right situations —such as part of a professional sometimes reveal themselves. Darling-Hammond et al. development initiative—also lead to improved teaching and stu- (2017), for example, selected a small set of high-performing dent outcomes (e.g. Lai et al., 2014; Van Geel et al., 2016). systems and then sought to examine commonalities and Various theories of change for why EIP should lead to generate useful insights. Germane to the present study, they improved teaching and student outcomes have been established found these jurisdictions tended to view and support teach- (e.g. see Brown et al, 2017; Cain, 2015; Cain et al., 2019). Broadly ing as a “research-informed and research-engaged profes- these argue that, assuming they have both access and the capacity sion” (p. 15). Relatedly, they noted systemic ways in which to do so, teachers can use a full gamut of evidence in relation to these systems supported teaching as a collaborative rather the decision-making that occurs as part of their work. For than an isolated occupation (e.g., affording opportunities to example: (1) evidence—especially forms of data—can be used by observe others’ lessons, fostering teacher sharing within- and school leaders to identify and pinpoint areas for improvement, across-schools). 2 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 ARTICLE There have also been a small number of more direct com- High social Low social parative examinations of EIP (or evidence-informed policy) in cohesion cohesion recent years, though these too have featured different samples, goals, and analytical approaches relative to our paper. For example, the Organisation for Economic Co-operation and Development (OECD) published an edited volume, “Evidence in The The fatalisc Education: Linking Research and Policy” (Burns and Schuller, High social hierarchist 2007). This book brought together “experts on evidence-informed regulaon way way policy in education from a wide range of OECD countries” (OECD, 2020, n.p.). The book as a whole, and chapters within, supplied numerous insights while detailing various ‘cases’ (e.g., organisations, knowledge brokers, programmes). However, its focus on evidence-use within systems was generally limited, it The The focused more at policy than at practical levels, and contributors Low social regulaon egalitarian individualist did not operate from a common framework to aid cross-case way way comparisons. Also, the Evidence Informed Policy and Practice in Education in Europe (EIPPEE; see http://www.eippee.eu/) net- work’s funded work yielded comparative information and insights relative to knowledge brokerage activities and mechan- isms across 11 European countries (e.g., see Gough et al., 2011). Fig. 1 The social cohesion/regulation matrix. This figure depicts a social Importantly, the analytical framework they developed and drew cohesion/regulation matrix, which draws on the work of Hood (1998). upon reflected systems thinking/modelling; in line with Best and Holmes (2010), they emphasised how various agents/actors are tied together by a system and embedded/organised through The second axis, ‘regulation’, refers to the institutions that structures that shape the interactions and knowledge exchange determine control and how accountability functions in a system. that ultimately takes place. Typically, in a high regulation system there is a dominant hier- Similarly, we see value in a systems approach. Given this, we archical culture with associated bureaucratic control. High reg- employ a dual analytical frame (as described below: a cohesion/ ulation systems often also involve the danger of ‘high stakes’ regulation matrix and institutional theory) to supply a metho- failure; i.e. a situation in which not meeting exacting account- dological lens through which to understand EIP within and across ability standards results in individuals or institutions being highly four systems. The system matrix, presented next, supports the penalised. Systems displaying low social regulation, on the other idea of the “power of context” (Chapman, 2019, p. 4). It high- hand, tend to exhibit much flatter, non-hierarchical cultures, with lights the critical role of the local diversity of settings that systems improvement achieved through partnership. A low social reg- operate in and contributes to understanding the variety of layers ulation system is also much less likely to have external account- intervening in the implementation of any school reforms and ability measures which lead to penalisation. innovations, including EIP. Besides, the matrix ensures there is a As can be seen in Fig. 1, combinations of high/low social focus on the different challenges systems experience, and repre- cohesion and high/low social regulation result in the following sents a starting point for any analysis which looks on the con- four system types (Hood 1998, p. 9): figuration of facilitators and barriers in any school improvement process. Each context is unique and powerful and determines a 1. The fatalist way (top right quadrant): characterised by rule- specific configuration of factors, agents, and conditions, which bound approaches to organisation but there is little can be explored in-depth through the lens of institutional theory. cooperation to achieve outcomes. Together, then, our dual approach not only provides a way of 2. The hierarchist way (top left quadrant): displays social accounting for the macro-level differences between contexts, it cohesion and cooperation in order to meet rule-bound also enables a comparison of meso-level and micro-level factors approaches to organisation. Often characterised by (via institutional theory) that might be common and distinct bureaucracy. across systems. 3. The individualist way (bottom right quadrant): atomised approaches to organisation and which involve negotiation and bargaining between actors. The cohesion/regulation matrix 4. The egalitarian way (bottom left quadrant): high participa- Globally, school systems have a range of differentiating contextual tion structures, in which all decisions are ‘up for grabs’ and structural elements. The cohesion/regulation matrix, set out combined with an egalitarian culture and peer to peer in Fig. 1, has previously been used by Chapman (2019) (drawing support. on the work of Hood, 1998) as a way of segmenting school sys- tems according to the principal macrolevel or system-level factors that define them. The axis used to form the matrix may be Institutional theory considered as follows: ‘social cohesion’ refers to the institutions, Within each school system there are further factors that affect the norms and networks that bind societies together. Systems with behaviour of school leaders and teachers. To account for these we high social cohesion have a higher propensity and willingness to draw on institutional theory. Institutional theory (Powell and collaborate. Threats to social cohesion—which tend to result in DiMaggio,1991) offers a useful and under-utilised lens for low socially cohesive systems—tend to occur when such struc- understanding the complicated relations between evidence and tures and systems (e.g. specific layers of government, the trade practice (Martin and Williams, 2019). Institutional theory is unions, the church, as well as the provision of universal services aimed at clarifying facilitators and constraints, within a given such as health) are dismantled and replaced with deregulation social field, to organisations’ behaviour. Accordingly, it can show and privatisation. In other words, approaches that place an onus how organisations’ (i.e., schools/districts) activities are governed on individual agency over collective approaches (Bauman, 2012). by various formal and informal rules and norms (Martin and HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 3 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 Williams, 2019). These rules and norms tend to be durable, and between them and lessons for improvement. We see a mosaic of within a field we tend to see substantial continuity—because, for policy landscapes across Europe and the U.S., each one with its example, different organisations are frequently subject to similar unique development, challenges, tensions, and dilemmas. The pressures and tend to have reached similar understandings Catalan model is an example of the contextualisation of legal regarding what behaviours are (in)appropriate within their realms regulations, being part of the Spanish system, characterized by (Powell and DiMaggio, 1991). collegiality, decentralisation, and institutional autonomy. Despite The ‘state’ is generally an important actor when considering the high level of social cohesion, local regulations have a central public service provision, as for example public sub-entities are influence, enhancing the level of bureaucracy and generating a typically at least partially dependent on the state for resources. hierarchical culture, which supposes a series of tensions between Accordingly, state-level expectations are influential, and some system levels and agents. The English system is typical of what are codified in formal policies that direct attention and work Pasi Sahlberg refers to as Global Educational Reform Movement effort in certain directions. Thus, certain forms of evidence— (Sahlberg, 2016) systems: as well as being characterized by high the focus of this manuscript—and certain priorities are likely to autonomy and high accountability, there is a focus in England on be privileged while others are side-lined or given lesser focus. In the core subjects of literacy and numeracy. This has led to much many cases (albeit more so in some fields than others), pro- standardisation of practice, despite the aim of recent reforms to fessionals also are significant in terms of enabling/constraining create an innovative self-improving system. In other words, and regulating members’ behaviours. Within organisations despite having the freedom to experiment, school leaders often some individuals are relatively more powerful than others, choose to emulate the practices of others out of fear of being an which also might have implications in terms of evidence use. outlier and subsequently punished for failure. Massachusetts For example, Brown and Malin (2017) described how school appears by most measures to be the top-performing state edu- principals are pivotal in terms of bringing about EIP in their cation system in the United States. It is also a context in which respective settings. Such individuals and groups are also often considerable efforts have been made, at state and national levels, key to knowledge brokerage in various ways (e.g., via setting to increase/enhance the use of research in policy and practice. For meetings, via their communications, and so forth; see Malin example, its state-level education department has shown a sys- and Brown, 2020). tematic and pioneering commitment to planning and research, A key insight from institutional theory—as applied to the and key federal law in the US maintains evidence and account- study of evidence use—is that “the strength of the evidence is ability requirements. RP is also unique, as it is the only German not the sole, or even the major, determinant of its influence on state without state-wide exams. It has justified doing so by practice; rather, more powerful actors hold considerable sway pointing to its students’ comparatively good results in nationwide in determining what (and indeed whether) evidence is used” tests of student performance. More recently, even the school (Martin and Williams, 2019, p. 55). However, “this does not inspection system has been abolished (the current study, however, also mean that there is no deviation from institutionally pre- refers to this instrument), as it was evaluated as too costly and not scribed behaviours” (Martin and Williams, 2019, p. 56). In fact, very effective. Instead, more investment is now being made in practitioners are often creative despite substantial constraining enhancing support systems for schools. RP thus offers a parti- institutional forces. Thus, a challenge for researchers using cularly good example of a rather egalitarian approach in the institutional theory is to sensitively examine contextualised matrix and data use in such a system. norms, rules, and structures, while also attending to the reality Despite our best efforts, this study contains certain limitations. of what is occurring in order to accomplish the focal organi- For example, our study relied upon authors’ access to extant data sation’s main tasks. and other information (versus requiring novel data collection) related to the focal questions. Accordingly, we did not as part of this research collect/analyse uniform data across the cases we Research questions and approach studied, but rather relied upon what was available for each case. In the following paper, we attempt to shed light on these three Accordingly, cross-case comparisons are made—and should be questions: taken—with some caution. The contexts we studied also do not represent the full diversity of educational systems internationally. 1. To what extent are teachers in systems with different types Still, our aim is to compare an educational phenomenon within of cohesion/regulation characteristics engaging in EIP? distinct cultural areas, with regard to its contextuality and dif- 2. What EIP-related enabling or hindering factors do different ferent governance constellations in the sense of international systems present and what are the relative ‘strength’ of these comparative education: “As comparative education is a field that enablers and hindering factors? is fundamentally grounded on an interest in learning from each 3. Are there generalisable lessons from education—at the other’s experience (that is, generated from each other’s contexts), system level—that can be applied to improve EIP in other context has always mattered“ (Lee et al., 2014, p. 150). For key areas such as health, justice and social care? example, in this study we did not include any systems that fit in To address these questions we present four school systems— the individualist quadrant of the social cohesion/regulation Catalonia, England, Massachusetts, and RP—as miniature case quadrant. studies. These cases were selected out of a combination of con- We have classified each context according to the cohesion/ venience and strategy. Strategically, the recruitment of this study’s regulation matrix in Fig. 1 and justify this by detailing the spe- team of authors was driven by the lead authors’ desire to com- cifics of the different systems that make this so. We then set out to paratively examine contexts reflecting diversity along the examine: (i) the extent to which teachers implement research dimensions being studied. Accordingly we sought authors known evidence into their teaching practice (outcome); (ii) which to possess information access and expertise that would collec- enablers and barriers with research use are described in the dif- tively enable us to address this study’s main questions, relating to ferent systems, using institutional theory as a way of guiding how the context with which they were most familiar as we turned later and why types of evidence are privileged and the more and less to a cross-case analysis. powerful evidence actors within those systems; (iii) finally we We selected cases that correspond to different system models, assess the relative ‘strength’ of these enablers and barriers linked creating opportunities to identify commonalities and differences to specifics of the different systems. 4 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 ARTICLE Catalonia (Spain) make scientific knowledge an engine for improving educational The Catalan educational model, mirroring the Spanish model, sits practices and policies. To do that, the programme called within the top left quadrant of Fig. 1 (the hierarchist way). It is “Evidence-informed schools” [Escoles d’evidència] aims to put characterised as a system based on a collegial model, decen- evidence on what works in education in the service of educational tralisation and processes of institutional autonomy, as well as the policies and schools, to promote the most rigorous empirical contextualisation of legal regulations (Marchesi and Martin, evidence, and at the same time to connect it with the needs of the 2002). It is a model that offers a privileged position to agreed- system, schools, and teachers. According to the Decret 274/2018, upon proposals for action based on an educational project and a with this programme, Catalonia will work towards the articula- series of intervention projects for improvement; which, according tion of an ecosystem that brings together all the educational to the Act on the Right to Education (LODE, 1985) currently agents and puts research evidence at the service of improving include: improvement plans, the curricular project, the environ- education. mental plan, etc. Schools display different levels of autonomy in Despite the late visibility of the concept and some pioneer the areas of planning, management and organisation. However, initiatives implemented in schools in recent years, (especially with a strong background as a centralized system in the 1990s, promoted by the private sector such as the EduCaixa programmes schools are responsible for designing and implementing educa- [EduCaixa, 2019] or innovation movements based on evidence), tion and management plans under the supervision of their one cannot say that this trend has become generalised. Studies respective education authorities. It is within the powers of the show that adopting an evidence-based view of teaching requires principal according to the Act on the Education Quality an understanding of how to integrate teachers’ experiential Improvement (LOMCE, 2013) to ensure the functioning of the knowledge and should be complemented by contextual and school and stimulate improvement processes. According to this experiential interpretations of research with a reflective approach model, Catalan educational law reinforces the idea of co- to practice (see Ion et al., 2019). This is a paradigm shift that responsibility between schools and local authorities in decision- involves interventions at all levels. making, promotes collective responsibility in management, and empowers school management teams and teachers themselves to promote innovation through a horizontal system that supports Application of institutional theory. Framed by institutional collaborative initiatives (Gairin, 2015). The model balances high theory, we can discern that different levels are involved, from an social cohesion with a high level of participation. At the same epistemological level linked to one’s conception of research, to the time, it addresses educational inertia with a high interest in personal, organisational and systemic levels (these are further accountability and governance through a weakly articulated explained below). The ecology of educational practices and policies structure based on a cultural model of quality assessment that that are ‘enriched’ (Oancea, 2018) with research should involve, in a uses rigid standards and structures. The coexistence of two shared manner, all actors: teachers, students, public administration, models of governance in tension makes it difficult to fully tran- local authorities, and government agencies. The dialogue among sition to a completely decentralised and cohesive model. these actors involves harmonising different interests and narratives, which in turn depends on, and generates power relations amongst, different contexts and tensions and dilemmas between contexts and Use of evidence. The Catalan educational system is experiencing levels, agents and decision makers. a very diffuse and spontaneous wave of change and educational First, at the epistemological level, Spanish teachers tend to innovation (Martinez, 2019), magnified by a clear commitment to disconnect the conception of research from educational practice. the emergence of collaborative networks between schools and For example, data indicates that teachers perceive research as a social and educational organisations (Azorín, 2019). Of late, this type of abstract knowledge that is useless, of poor quality and far phenomenon has been driven by the emergence of complex removed from their daily practice (Murillo and Perines, 2017; problems and the lack of sufficient resources (Díaz-Gibson et al., Murillo, 2006; Díaz Costa, 2009). The ability to support research- 2015) to address them, leading schools to seek solutions through informed practices from the bottom-up depends equally on the ‘Planes Educativos de Entorno’ (environment educational teachers’ individual capacity to promote and work in a climate of plans) or ‘Local Educational Networks’ (Civís and Longás, 2015). trust and collaboration, in which the exchange of knowledge These experiences are based on collaborative work and networks and the shared and grounded construction of and critical that assume the existence of a new paradigm for socio- reflection on their own practice, represents their own professional educational services and the growth of extended (community) ethos. However, studies in the Spanish context show that schools (Azorín, 2019). These developments have generated a innovations are rarely based on research evidence, although complex scenario with different levels of involvement and teachers themselves claim to be in favour of the use of evidence in improvement and specific adaptations as a function of contextual their classroom practice. In a study conducted with teachers in variables such as school ownership (public–private), teachers’ Madrid and Catalonia, 68.1% of teachers and 77.3% of principals stages of professional development and attitudes towards change declared that they frequently or always use research to inform and innovation (Perines, 2018), to mention just a few; these their practices (Ion and Gairín, 2019). However, when they have variables can generate an imbalance between levels of social to inform their innovations in class, teachers acknowledge limited cohesion and commitment to change. use of scientific evidence in favour of experiential and peer To face these challenges, in recent years, so-called ‘evidence- knowledge (Perines, 2018; Ion et al., 2019). Among the factors informed practices’—which involve teachers integrating research that limit the development of the ability to use research, teachers evidence into decision-making—has taken on increased impor- include limitations of time, resources, or support from the tance in the practice of schools as well as having a greater management team (Perines, 2018). In addition, teachers identify visibility in the public discourse. With the adoption of the Catalan clear deficiencies in their initial research training and a strong Education Act (Decret 274/2018), a systemic and formal disconnect with the context of the production of research, marked commitment to the promotion and use of evidence and research by concerns about issues that diverge from the reality of the in the field of education has been put into place. This classroom (Perines, 2017). commitment represents the beginning of a new stage in Catalan At the organisational level, the development of the capacity to policymaking promoted by the public bodies, where the aim is to use evidence requires leadership that is clearly sensitive to HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 5 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 research and favours a positive organisational culture (Ion et al., extend their reach to support other schools as they look to 2019). However, this is not sufficient to promote the use of improve (Greany, 2014; Robinson, 2017). evidence unless it is accompanied by a research culture that A key point to understand is the structure of England’s school supports a general orientation towards the use of evidence in any system. While education policy is shaped centrally, since the early decision-making process, assumes the contextual nature of 20th century local authorities had responsibility for the education knowledge, supports the integration of evidence into teachers’ of children in their locales. However, the relationship between professional development and cultivates an organisational ethos local authorities and central government has not been easy and in favourable to collaboration and academic integration (Oancea, 1988 the Education Reform Act saw local authorities lose many of 2018; Ion and Gairín, 2019). their powers until their role was one of scrutiny and support. A At the system level, promoting a vision of practice based on significant change came with the establishment of academies, evidence requires coherent and responsible actions among all state schools directly funded by the Department for Education actors. In Catalonia, innovative educational initiatives are still far and outside local authority control. As self-governing trusts from being a generalisable trend; rather, initiatives are isolated academies have a number of freedoms afforded to them in terms and depend on personal initiative (Camacho, 2016). This of innovation and curriculum which local authority schools do undoubtedly contributes to the fact that efforts to promote not. While some academies operate independently, a number of improvement and innovation continue to be poorly recognised these schools are networked into trusts, groups of schools with and rewarded, poorly documented (Perines, 2017, 2018) and very centralised policies, curriculum and approaches to professional diverse. Furthermore, despite a high level of thematic diversity, learning. Therefore the type of autonomy that staff in schools initiatives appear to be minimally connected and are quite experience will vary depending on the type of school and its different at the methodological level. Additionally, at the system structure. level, there are few mechanisms to identify and connect To further encourage improvements in quality and innovation, innovative educational practices with one another, which makes policy makers have also embedded a range of accountability it difficult to identify different agents’ degree of development and systems. These “combine quasi-market pressures—such as par- involvement in the progress of the innovation process (Camacho, ental choice of school coupled with funding following the learner 2016; Martinez, 2019). In Catalonia, in a context of increasing —with central regulation and control” (Greany and Earley, 2018, concern based on accountability (Catalan Government, Departa- p. 7). A key aspect of this system is the regular school inspections ment d’Educació, 2019), politicians dedicated to managing process undertaken by Ofsted (the school inspection agency in educational systems tend to have a reduced view of educational England). Ofsted inspections are highlighted by many school research, such as the evaluation of educational systems. leaders as a key driver of their behaviour and for good reason. As Additionally, the approval of laws and decrees are often not a result of an inspection, for which there is less than 24 h notice, sustained by research, and the existence of accumulated research schools are placed into one of four hierarchical categories of (such as research syntheses or meta-analyses) is simply unknown grades. The top grade—‘outstanding’—has historically carried a (Martinez, 2019). In this way, a divided model is reproduced, in number of benefits. For example, it makes the school more which decision-making is separated from enquiry and reflection attractive for parents, meaning more students apply to attend, on practice, and the two exist in remote spheres that do not and thus more funding is directed towards the school. The reverse respond to each other. Change involves: (1) making evidence is then true, that schools with lower ratings find it more chal- both sides of the same coin of discourse and practice (i.e., political lenging to attract families and the attached funding with it. In discourse supporting evidence in practice and the measures taken addition, up until 2019, schools rated outstanding were exempt should be aligned, to ensure coherence between all layers and for subsequent inspections (even with changes of leadership and actors); (2) introducing research as an instrument of both the staff), meaning that accountability pressures are considerably political system and of governance; and (3) creating the lessened. At the other end of the scale, schools judged to be in the conditions for research to fulfil a social function; that is, to have lowest Ofsted category—‘inadequate’—are subject to a forcible an impact beyond the academic function. Political discourse must removal from local authority control and the Department for double the measures to promote governance mechanisms that Education pay academy trusts to take on these schools in a bid to stimulate this type of practice and that support it with adequate rapidly improve performance. Inspection frameworks have seen a resources and mechanisms of recognition and reward for number of changes in them, which directly impacts the work in horizontal and collaborative initiatives. schools so that they are meeting an ever-evolving criteria of what is considered by Ofsted to be good practice. In addition to inspection is the use of government produced England (UK) annual ‘league tables’ of schools and a publicly available ‘Find and England’s high accountability (high social regulation) and high compare schools in England’ website, which allows those acces- autonomy (low social cohesion) context places it firmly in the top sing it to rank schools on a number of different variables and right-hand quadrant of the matrix (the fatalist way). These two student outcomes. As a result, it is acknowledged that England’s elements have achieved particular prominence in the last decade. accountability framework both focuses the minds of—and places Beginning with the latter, central government policy makers in pressure on—school leaders to concentrate on very specific forms England have now devolved multiple decision-making powers of school improvement. In particular, such improvement prin- and resources to schools. Included in this process of devolution is cipally tends on ensuring students achieve well in progress tests in the responsibility for teacher professional development, in the key subject areas (e.g. English literacy and mathematics) (Ehren, belief that this will improve quality and increase innovation 2018) leading to a narrowing in the curriculum. (Greany and Earley, 2018; Howland, 2015). This commitment has been described elsewhere as the move towards a ‘self-improving school system’ (Greany, 2017). Here the characteristics of ‘self- Use of evidence. Some data does exist in terms of EIP in the improvement’ include individual schools now having greater English context. For example, a survey of 1670 teachers in Eng- responsibility for their own improvement; that teachers and land was undertaken by the National Foundation for Educational schools are expected to learn from each other so that effective Research in 2017. Here it was found that academic research had practice spreads; and that schools and school leaders should only a ‘small to moderate’ influence on teacher decision making. 6 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 ARTICLE Instead of research-evidence, when deciding on approaches to 2015). In addition, the ability for school leaders to put in place improve student outcomes, teachers were in fact much more structures within their school to enable teachers to engage likely to draw ideas and support from their own experiences (60% effectively in collaborative EIP development have been limited by of respondents identified ‘ideas generated by me or my school’), the budget cuts which have ravaged the education and wider or the experiences of other teachers/schools (42% of respondents public sector in England (e.g. see Busby, 2019). Teachers identified ‘ideas from other schools’). In addition, non-research- themselves also lament the lack of time they have to do anything based continuing professional development (CPD) was also cited as other than their day to day role, with EIP often seen very much as an important influence (54% of respondents). These compare to a luxury (Brown, 2020; Galdin-O’Shea, 2015). This argument is the much lower figures of 13% and 7% for ‘sources based on [the] reinforced by OECD data which indicates that England’s Primary work of research organisations’ and ‘advice/guidance from a teachers have the fifth highest number of teaching hours out of all university or research organisation’, respectively (Walker et al., countries surveyed. While, a teacher in Finland has 677 h—and in 2019). The survey also asked teachers to identify the relative German they have 799 h of contact time with pupils—a teacher in importance of a range of factors likely to have an impact on any England has on average 942 h (OECD, 2020). decision to adopt a new approach to teaching and learning. The factors that teacher respondents were most likely to identify were: straightforward to use (47%); aligned with professional expertise Massachusetts (USA) (46%); and a good fit with existing practices (44%). Only a third Placing the Massachusetts (MA) United States (U.S.) school (32%) indicated research was a ‘strong influence’ on their decision system in the cohesion/regulation matrix is not entirely to adopt their approach (Walker et al., 2019). straightforward. Regarding the individualism-egalitarianism dimension, though MA and the U.S. are individualistic in nat- ure, teachers in their work settings are often more communitarian Application of institutional theory. Turning now to institutional (Shober, 2016). Most recently, however, MA has embraced neo- theory and it is clear that some salient elements are present. For liberal and managerialist education policies (Piazza, 2017; Hors- instance, EIP has been supported from non-governmental orga- ford et al., 2018); for instance, in 2012 MA enacted a law that nisations that operate to support schools. The Education limits seniority-based job protections for teachers and may Endowment Foundation (EEF)—the ‘what works’ centre for undercut a communitarian, professional ethos. (Also see later education in England—for example, provides a freely available discussion of Race to the Top policy.) MA is also—relative to ‘tool kit’ of what works evidence in order to ensure summaries of other U.S. states—socially cohesive (e.g. Wise, 2015 places it in educational research are accessible to non-academics. In addition the top 10 on this measure). However, the U.S. presently is to this substantial investment, in 2014 the EEF launched a £1.4 m conspicuously un-cohesive, and MA is no marked exception. All fund to improve the use of research in schools (EEF, 2014). This considered, we have placed MA (like England) within the top initiative was followed up in 2016 with the launch of the EEF’s right quadrant, the fatalist way, albeit with the understanding that Research Schools initiative; schools charged with leading EIP in this is a dynamic context and policy area. their local area. There has also been a substantial rise in bottom MA is unique, relative to other U.S. states, in that its K-12 up/teacher-led initiatives, such as the emerging network of students’ achievement consistently ranks at or near the top ‘Teachmeets’ and ‘ResearchED’ conferences (Whitty and Wisby, (Papay et al., 2020). It has also, since at least 1993, been viewed as 2017) designed to help teachers connect more effectively with a leader in U.S. education reform. In 1993, the omnibus Massa- educational research. Furthermore, a prominent example of a chusetts Education Reform Act introduced state-wide learning teacher-led initiative was the 2017 launch of England’s Chartered standards and an associated state testing/accountability system. In College of Teaching: an organisation led by and for teachers and exchange for the increased accountability, state-based school whose mission, in part at least, is to support the use of EIP funding also considerably increased. Notwithstanding these (Whitty and Wisby, 2017). EIP is also being increasingly pro- efforts and successes, educators and policymakers in MA have moted and supported at a government level. For example, Eng- also taken note of, and have sought to rectify, considerable per- land’s Department for Education funds the work of the EEF, and formance inequities between certain groups of students (i.e., has also ensured the inclusion of references to EIP within prin- inequitable outcomes according to class, race, ethnicity; Darling- cipals’ standards and in the pilot Early Career Framework for Hammond, 2010; Papay et al., 2020). newly qualified teachers. Finally, the periodic Research Excellence In the U.S., as the national constitution does not specifically Framework (the ‘REF’), via which UK universities are funded, address education, states hold primary authority, though states now requires them to account for the “impact” their research has historically have also delegated much authority and responsibility had on, “the economy, society, culture, public policy or services downward—i.e., to school district levels. Thus it is appropriate to … beyond academia” (Higher Education Funding Council, Eng- examine U.S. school systems at a state level, but with attention land (HEFCE), 2011, p. 48). In other words, the government’s also to local variation (i.e., districts, schools, teams). Nevertheless, aim is to use REF to encourage universities to ensure that their educators are also substantially enabled/constrained by federal research is used in the world beyond academia, for example by policies. U.S. educators have since the early 2000s needed to directly working with teachers and schools (Cain et al., 2019). respond to a largely federally led “what works” agenda, char- Yet from the figures above it is clear that EIP has some way to acterised by a “strikingly narrow focus on evidence of the impact go before it becomes a way of life in schools in England (Bell of interventions” (Tseng and Coburn, 2019, p. 351) and et al., 2010; Walker et al., 2019). Furthermore, the take-up of the neglecting broader concerns and types of evidence. Most notably, EEF toolkit is limited, with just less than a quarter of teachers the No Child Left Behind Act of 2001 (NCLB) introduced high- indicating they accessed it. The figure for school leaders is much stakes student achievement testing mandates and required that higher however at just under 60% (EEF, 2018). Reported barriers school leaders and teachers reduce achievement gaps. Although to EIP are manifold and include research held behind paywalls, NCLB stimulated educators’ data use, it also squarely emphasised dense academic style writing which can be difficult to access, summative measures and contributed to other dubious practices underdeveloped research literacy skills and support, both (e.g., narrowed curricular offerings and a focus upon “bubble individually and organisationally; and the pressures of high kids” who tested near proficiency cut-points; Datnow et al., 2013; stakes accountability in England’s schools (Brown, 2017; Greany, Hackmann et al., 2019) HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 7 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 In 2010, MA pursued and was subsequently awarded a US formal research questions and data collection/analyses. Thus, $250 mm federal Race to the Top (RttT) grant. RttT was an most common—at least among this sample—is engagement in Obama-era federal grant competition (the largest ever of its informal research related to several key areas (e.g., tracking school kind in US history, and brought forth on the heels of a recession culture and climate). that left states especially solicitous for funds) that was aimed to Research evidence use reportedly occurred in certain ways—for spur educational innovation, but that was specifically focused instance, 25% noted looking at the research base as part of around stimulating particular state-level reforms, including: (a) selecting new programmes or interventions. Respondents also adoption of core standards and assessments; (b) building data reported using either data or research to “adopt new materials” systems that could measure student success and inform tea- (9.13 on a scale of 1–10; 10 = all the time), “select intervention” chers/schools how they could improve; (c) the recruitment, (8.87), “provide professional development” (8.27), “inform development, and retention of effective teachers; and (d) instruction” (8.07), “allocate funds” (7.53), and “allocate staff” turning around low-performing schools (Horsford et al., 2018). (7.0). Evidence was also used by some respondents/districts to MA and other states needed to develop and implement policies measure implementation, and to measure impact (primarily via in these areas, which clearly reflected a neoliberal approach to student assessment data). education reform, in order to compete for and receive this This study also inquired about real and potential barriers to funding. These policy shifts subsequently enabled and con- evidence use. Responses suggest limited time/staff resources (12 strained educators’ behaviours and focal areas, introducing new mentions), value of available research (5), and culture (3) present regulatory pressures and directed their attention toward certain the largest obstacles. Confidence in engaging with research did forms of data and evidence (more discussion to follow as part of not appear to be a considerable barrier, at least for these institutional theory analysis). respondents. Across-district evidence sharing appears to be MA’s educational system is hierarchically organised, albeit in modest amongst MA educators, with most such sharing occurring some cases with overlapping authority and in some aspects with “at conferences or collaborative meetings” (p. 9). Hedberg (2018) the higher (i.e., state department of education) level serving more globally sensed some “scepticism around research” (p. 13) as a resource/support (and less as a heavy-handed governor) to including vendor-produced research that was thought to be the local districts and their educators. For example, and pertinent biased. to this manuscript, MA’s state-level education department was Lastly, it was clear that most educators accessed research the first to include a state research director, part of a robust Office indirectly and in brokered fashion—i.e., through professional of Planning and Research (OPR). MA educators and state edu- associations (8 mentions) and conferences, and from ‘other cation officials are also beholden in some key ways to federal education publications such as the Marshall Memo (also see educational law, such as the Every Students Succeeds Act of 2015, Malin et al., 2018; Malin and Paralkar, 2017). State resources were which replaced NCLB and, while devolving some authority to also noted, again underscoring the importance of the broader states, largely kept intact key evidence and accountability system in facilitating or hindering use. requirements (Tseng and Coburn, 2019). Application of institutional theory. When applying institutional Use of evidence. What does all of this mean in terms of MA theory to the MA case, some key elements are evident. First, we educators’ engagement with research evidence? This case draws can see how formal policies (state and federal) have constrained primarily upon findings presented as part of a recent study, educators’ attention in certain directions (e.g., attentive to high- ‘Evidence use in Massachusetts School Districts’, completed by stakes testing data, and more generally toward data relative to Hedberg (2018) for DESE’s OSR. This interview study (N = 22 other sources of evidence). Within that parameter, we can see the district-level interview participants), drawn from a stratified state attempting to be helpful, providing data in a sample of MA districts, ‘sought to understand how districts are timely fashion and in a format that is said to be desirable. There currently using, building and sharing data and research’ (p. 1). also appear to be earnest and somewhat successful efforts to This study was undertaken as part of early stage efforts by DESE/ facilitate coherence across the system, with the state assuming a OSR to support MA districts’ evidence use. Supportive insights leading role. The pattern thus appears to be a primarily top-down are also drawn from a piece by Carrie Conaway (2020), who then approach to evidence (and, more specifically, certain canonical was research director for OPR. data), whereby evidence from some other authoritative location is The first point is that ‘data’—and especially from a particular brought into practice (Martin and Williams, 2019). The state source—are being used more frequently than research. In part, also is showing interest in encouraging/facilitating bottom-up this finding reflects the aforementioned policy and larger agenda, use, e.g., by “facilitating the cross-pollination of ideas and which has tuned educators’ focus especially toward students’ resources by evidence-oriented practitioners” (Hedberg, 2018 performance on annual tests. Indeed, in this case educators recommendation, p. 2). mentioned their state test—“MCAS”—data as being of premium Despite considerable isomorphic pressures on MA educational importance. Hedberg observes, “looking at student performance organisations and educators, from the top, there also appears to data is becoming part of the regular routines for teachers and be considerable organisational-level diversity in EIP. Some administrators” (p. 4). Some districts (N = 6 of 19) have also districts, for instance, have invested in data coaches or similar, invested in one or more staff members whose function revolves whereas others have not. Meanwhile, some schools and districts around data or evidence. In all, districts appear to “have more are facing major pressures to improve or turnaround their schools systems for integrating data use in their decision-making than for (again, a top-down policy), and educators within these districts integrating outside research” (p. 9). These systems/structures are compelled to engage with evidence in different ways and with include data meetings (13 mentions), professional development different levels of urgency. Within such a situation, we suggest (5), and dedicated data teams (5). there is still potential for cross-district sharing if/when educators/ In terms of ‘building evidence,’ findings were mixed. On one organisations could network and form consortia according to hand, many districts reported conducting in-house research and/ common challenges and interests. However, results of the or partnering with outside organisations in order to do so. On the evidence use survey suggest much of this potential is currently other, in most cases the descriptions offered did not suggest untapped. 8 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 ARTICLE We can also see that the state, though a very important part of related journals. On the other hand, the use of data generated in the evidence use system in several ways (e.g., via relevant laws, its the context of external instruments (feedback from school assessments, and its research supports), is not the sole influencer inspections, state-wide tests) is significantly lower (bottom left beyond the level of the organisation. For example, professional quadrant).1 It is noticeable that the approval ratings of school associations play a key role in facilitating evidence sharing and management members tend to be higher than those of the tea- use in MA, and certain other brokers/mediators are performing chers (not illustrated here). This might be explained due to important linkage functions as well. At the epistemic level, too, principals’ more comprehensive view of the school as an orga- educators’ scepticism toward research serves as a barrier to use, nisation, but requires further analysis. but the level of scepticism most likely varies considerably In addition to this data, Muslic (2017) shows in her study (a conclusion reached by Hedberg, 2018, p. 2, whose recommen- concerning the German federal states of Berlin and Baden- dations relied upon leveraging and connecting educators who are Württemberg that in many schools the subject-specific depart- more “evidence-oriented”). Overall, it would seem EIP in MA is ments are central to the discussion of the results of state-wide skewed top-down in important ways, but there is also recognition assessments. The measures derived include, in particular, process- of and are some earnest efforts also/instead to promote more related activities such as support concepts and didactic agree- bottom up EIP in and across MA schools and educational ments. An in-depth analysis of the results with regard to the organisations. Such bottom-up efforts, however, typically do not teaching quality, however, rarely takes place. The exchange include the conduct of formal research. between teachers and school management remains largely informal and rather superficial. Only in the case of poor results would the school management intervene. In many cases it is up to Rheinland-Pfalz (Germany) teachers themselves whether to use evidence and they are hardly The German school governance model can be classified in the accountable (low social regulation). quadrant of the egalitarian way. This means rather high social As the results from RP show, the new assessment and cohesion with lower institutional autonomy, where the idea of evaluation instruments are not only little used overall, but are ‘managing’ development processes encounters a still bureaucratic also regarded as comparatively unhelpful. Here the distance to administrative context. Meanwhile the German system is char- teaching practice may seem bigger than with other instruments, acterised by low social regulation; accountability is relatively low and the descriptive data usually do not provide any explanation stakes. This is the result of a longer development process. or knowledge of change. Furthermore, teachers perceive results, Until the 1990s, the German school system—with its 16 e.g. from state-wide testing, in particular as a starting point for country-specific variants in the federal system—was characterized considerations on quality development if they have an objective by an input-oriented governing model with hierarchically orga- reference norm, i.e. if there is information about what students nised school supervision and top-down detailed control through can do and in what respect they still have to develop competences laws and decrees. Comparable material and personnel resources (Kühle and van Ackeren, 2012). Longitudinal data with individual and the binding nature of the curricula were seen as a guarantee reference standards should also lead to more acceptance. for the quality and comparability of school results. This corre- The effectiveness of data-based school development improve- sponds to the logic of conditional programming (Luhmann, 1970) ment measures also requires a competent, trained handling of in the sense of standardising the results of work by standardising evidence-based knowledge and its integration into reflective the framework conditions. practice. Studies on the different use of data feedback in the As a result of the overall weak German results in Large Scale context of state-wide assessments in Germany show that schools Assessments and especially since the PISA study, orientation with high expertise in school improvement can benefit more from towards a logic of goal programmes (Luhmann, 1970) is now also data-supported feedback (so-called Matthew effect of accumu- found in more output-oriented steering and control elements. lated advantage) than those schools that have less competence in This includes the expectation that schools will orientate them- this area (Maier et al., 2011). Therefore, more systemic effort is selves more towards educational standards and that they will be needed to support schools’ development processes. accountable within the framework of external evaluation. This has led to a paradigm shift in Germany—in line with the interna- tional trend—although elements like competition and the market Application of institutional theory. Since 2006, following the so are not yet apparent or are only beginning to emerge. called “PISA-shock” stating unexpectedly weak results for Ger- many, the introduction of a nationwide educational monitoring Use of evidence. In the context of the joint project EviS (Evi- concept and data provision on school quality has been associated dence-based School Development), evidence-based knowledge with the hope of making the actions of schools more effective and and action in schools in the German federal state RP has been thus contributing to the improvement of schools. In the mean- operationalised and descriptively analysed. Evidence was defined time, education policy seems disillusioned with regard to com- as systematically generated, objective and explicit information on parative measurements and data use. the effectiveness of educational processes (Demski et al., 2012). On the system level, the usefulness of the data for school and The spectrum ranged from scientific empirical studies as well as teaching improvement has been questioned by the Standing state-wide assessments and school inspections (external evidence) Conference of the Ministers of Education and Cultural Affairs of to peer observation and student feedback (internal evidence)— the Länder in the Federal Republic of Germany (e.g. Kuhn, 2014). regarding any sort of information as evidence if it is more or less There is also increasing debate on how empirical knowledge can objective, reliable, and valid (Dormann et al., 2016). Figure 2 be better integrated into education policy, administration and illustrates the reception and use of different evidence sources by school practice in order to achieve meaningful change. From a teachers (N = 1230) in the EviS-study (van Ackeren et al., 2017). German perspective, education is not a directly steerable or The data show comparatively intensive use with regard to internal controllable, technocratic production process (cf. on the German process-related information sources that are closely related to the school system van Ackeren et al., 2015; Secretariat of the Standing teaching practice of teachers, e.g. on the basis of systematic stu- Conference of the Ministers of Education and Cultural Affairs of dent feedback on teaching (upper right quadrant in Fig. 2). The the Länder in the Federal Republic of Germany, 2019). In this results also point to a relatively intensive use of school subject- context, state-wide assessments, school inspections and central HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 9 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 5.0 Data use high 4.5 Student feedback with 4.0 regard to teaching 3.5 Subject specific journals Lesson improvement Newspapers/ strategies within the magaziens with ed. school 3.0 Focus Data recepon Centralized tesng data high low Evaluaon of school School inspecon da 2.5 stascs Interdisciplinary… Centralized task collecons 2.0 Internaonal assesment (PISA, Tesng data collected PIRLS etc.) Statewide assesment by the school 1.5 1.5 2.0 2.5 low 3.0 3.5 4.0 4.5 5.0 Fig. 2 Reception and use of different evidence sources by teachers (N = 1230). This figure depicts the reception and use of different evidence sources by teachers, drawing on findings from van Ackeren et al., 2017. final examinations in Germany are still low in sanctions and school situation (Fend, 2009), gain expertise in the assessment gratuities for schools or teachers. and interpretations of own data and—most importantly—draw The partially critical policy view on continuous performance adaptive conclusions and strategies that lead to measurable measurements and EIP in schools is also influenced by the above- success in schools (Bremm et al., 2017). Accordingly, there is a mentioned research findings concerning the organisational level. In need for a new balance between gaining knowledge and Germany a ‘real’ and meaningful data-driven change is rarely supporting schools in the reception and use of data, including a initiated. With reference to “school development through insight” stronger focus on the action level (teaching) and the control level (Kotthoff et al., 2016, p. 338) rather than control or competition, it (school management). Corresponding competences, such as will be up to the schools in the end to decide how to react if integrating empirical results into reflective practical experience, standards are not achieved within the low stakes environment. should be included in teacher training, which has rarely been the There exists an understanding of school as a learning organisation, case so far in Germany. School leadership plays an important role which has been partially strengthened in its scope for decision- for harnessing the benefits of data use (Brown and Flood, 2019). making and action. Schools should be able to adapt to local changes German studies have shown that directive or discursive leaders continuously and with the help of data and be able to monitor their are more likely to favour the discussion of data within school, actions and the effects they have achieved themselves. while delegated school management is more likely to inhibit these Nevertheless, the coupling of the development of the overall processes (e.g. Kronsfoth et al., 2018). School principals can also system and the individual school is complex, since German positively influence an organisational climate that is open to schools and the individual actors in them can decide quite evaluations and instruments of performance measurement, as independently how to deal with external interventions; from the well as promote an overall organisational embedding of the individual perspective, the interpretive sovereignty over ‘school processing and use of evidence (Muslic, 2017). quality’ lies with the pedagogical professionals (Klein and Bremm, With regard to the current state of discussion in Germany, it can 2020). Overall, there seems to be a lack of fit between a demand be stated that there is generally an unsatisfactory situation of a cost- for science-oriented reflection from the outside perspective and intensive system for data generation without systematic utilisation an experience-based practice within schools. The data also seem impact. Therefore the ‘Standing Conference of the Ministers of to lack ‘cultural significance’ in relation to the specific context of Education and Cultural Affairs’ (KMK) has changed its overall the individual school and individual teacher (Heinrich, 2015). monitoring strategy for education in order to support a more Furthermore, school authorities often do not see themselves as transfer-oriented development through more relevant and expla- managers or at least do not take on this role in practice (Klein and natory knowledge for administration and schools and to process Bremm, 2020). This can be seen in Mintrop (2015), for instance, research findings more systematically as well as to better support who speaks of a “public management reform without managers” schools (Kultusministerkonferenz, 2015). It remains questionable, (p. 790) in Germany. In this way, pressure and unintended side however, whether a more binding requirement for practitioners’ effects are largely avoided, but change is not systematically data use is needed, for example regarding comprehensive and initiated and supported. evaluated quality management systems in schools and by It seems helpful to provide longer-term support, e.g. within the strengthening the role of supervision and leadership. framework of school networks and individual school improve- ment support. Furthermore, findings from design-based school improvement projects point to the relevance of different Discussion opportunities for schools in this context: to understand and re- In this study, we set out to further examine the question of how to contextualize externally produced data in their specific individual bring about more consistent, EIP in education. To do so, we 10 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 ARTICLE undertook a novel approach, examining EIP comparatively across report infrequently relying upon research to guide their practice? four geographical contexts (Catalonia, England, Massachusetts, Here, some sector-specific analysis/understanding is supportive. For and RP) through the use of particular theoretical perspectives/ instance, as Cohen and Mehta (2017, p. 649) observe, teaching has frameworks. We applied a cohesion/regulation matrix and insti- across many (though not all) contexts “[failed] to crystalize as a full- tutional theory to frame our analyses of factors influencing evi- fledged profession”, a feature that renders teaching/learning “vul- dence use across these contexts. Our aims in doing so were nerable to lay views of education and reform, as well as to inherited twofold. First, we hoped to generate provisional insights related to patterns of practice.” Education as a field is challenged by manifold fostering more/better EIP in education. Second, we aimed to and often competing/conflicting goals, multiple constituencies and achieve and share process insights related to this undertaking (i.e., often fragmented systems (Labaree, 1997), which “complexifies the to inform those who might wish to emulate and improve upon idea of a unified body of research informing classroom practice” these approaches). Lastly, given the aims of this special edition, (Lubienski, 2020, p. 183). Indeed, as compared to professions like we sought to ascertain whether there are generalisable lessons in law or medicine, education has fewer agreed upon truths and this study (either in terms of findings, or process) that could be shared definitions of problems and solutions (Willingham, 2012). In applied to improve EIP (and/or its study) in other sectors. part, this is also a reflection of the inherent complexities and con- First, reflecting on our research process, here we offer several tingencies around teaching and learning. Altogether, one might observations. Most generally, our diverse author team found the better understand why educators who have considerable freedom dual analytical approach to support our aims— it directed our may not direct it in consistent easy to spot ‘research-informed’ attention in common (albeit not entirely overlapping) ways, ways. As such, education is indeed somewhat unique in terms of providing us with structure through which to make sense of the featuring relatively weak and conditional research-practice links level and nature of EIP that existed in these contexts. More (Lubienski, 2020). specifically, the matrix offered us the chance to understand the Still, there are certain paths forward toward bringing about uniqueness of each educational system and its specific national/ more/better evidence use in education. In education (much like in local flavours, while the institutional theory provided the chance other fields), it helps to recognise that research evidence is but to find commonalities between these systems. Indeed, the inter- one of the potential influences on practice (Farley-Ripple et al., section of these two axes of analysis provided an understanding of 2018). Indeed, as Cain et al. (2019, p. 1074) summarise, there is enablers and barriers to EIP, as well as agents involved in each now “near-universal agreement that research-generated insights one of them. Accordingly, the dual frame has been (and can be) are an insufficient basis for practice”. Above all else, educators— helpful for accurately diagnosing key aspects and levels. In turn, like other practitioners and policymakers—want to be able to we suggest decision-makers might be able to apply the frame confidently make decisions about problems/issues that are (and/or insights derived from others’ applications of it) to important to them. Ultimately, it is realistic first to understand “develop appropriate context-specific rather than one-size-fits-all research knowledge represents just one of several forms of packages of support to stimulate improvements” (Chapman, knowledge educators might draw upon as they go about their 2019, p. 5). work. Accordingly, its professional use is not pre-given but is Nevertheless, in this instance, we were hampered by certain contingent on a host of favourable features and conditions, set challenges, which mostly can be understood as study limitations across multiple levels. As such, and although none of the cases that can be overcome by others. For example, we experienced addressed in this study represents a utopia in terms of profes- unevenness across cases in terms of the level and depth of data sional evidence use, individually and collectively they are assistive and evidence available for analysis. Conceivably, a future study in drawing out some such features that can help move educa- could utilise the same frame, but could proactively collect com- tional systems toward more desirable states. mon data across contexts—and could do so at multiple levels of The England and MA cases, for example, show how strong the respective systems—rather than relying, as we did, on extant accountability pressures (via inspection and high-stakes assess- data and literature related to the contexts. We expect such an ments, respectively) certainly can coax educators to consistently approach would further the comparative process. Also, and in focus on particular forms of data and research. MA educators are line with Martin and Williams’ (2019) advice for scholars in these particularly attentive to students’ performance on annual high- areas, we found institutional theory to provide a quite useful and stakes state tests, while English educators are considerably driven illuminating, if also complicated, lens for analysing and under- by official school inspections. However, there can be considerable standing the relationships between evidence and practice in (and costs associated with such approaches, roughly summarised as across) contexts. Although ultimately beneficial, it required sub- ‘what gets measured matters’ (and the reverse: what is not mea- stantial work for us as researchers to develop a shared under- sured might be underemphasised or ignored, reduced/cut from standing of the various elements and how they applied. In this programming, etc.). Accordingly, we offer that when account- aspect as well, though, we strongly suspect a pre-planned, active ability systems are in place, specific details (e.g., assessment areas, comparative study of evidence-use, using this dual approach, format, foci, speed, and quality of feedback) are salient. RP offers would go far in terms of understanding EIP variation (and, thus, a useful point of comparison here, in that their external assess- suggesting potential approaches to improve EIP). Further, we ment data appear to be attracting substantially less educator suggest such research endeavours would be useful beyond edu- attention; this fact is most likely explained by the relative weak- cation, or perhaps could support the simultaneous study of more ness of sanctions and gratuities associated with these measures. than one sector; these lenses can be illuminative irrespective of Depending on one’s vantage point, this might be construed as a sector, as nuances related to norms, policies, traditions, etc. virtue or a vice. On the positive side, for example, perhaps RP (beyond mere differences in evidentiary bases across sectors) will educators are freer to direct their attention toward locally go far in terms of understanding EIP patterns/variation. important data and research (e.g., bottom-up evidence genera- Beyond the process, we also suggest this study’s findings yield tion/use), and as such are utilising the external data only to the some tentative insights in terms of EIP, both for educators/edu- extent that it is perceived to add value to their decision-making. cational scholars and for those outside the education area. Par- On the other side, results show that low stakes accountability and ticularly for the latter group, certain facts may seem higher degrees of autonomy in Germany come with lower levels counterintuitive. For example, why is it that, in a system in which and less infrastructure for support that would help practitioners teachers have a great deal of autonomy like Catalonia, teachers with understanding, discussing and recontextualizing these data. HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 11 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 The MA case also shows a relatively strong and layered Bell M, Cordingley P, Isham C, Davis R (2010) Report of professional practitioner infrastructure for supporting evidence use, including a state-level use of research review: practitioner engagement in and/or with research. research director and department focused on planning and CUREE, GTCE, LSIS & NTRP, Coventry. Available via Curee. http://www. curee-paccts.com/node/2303. Accessed 24 Aug 2019 research. Moreover, with their recent report, they too are now Best A, Holmes B (2010) Systems thinking, knowledge and action: towards better positioned to be more evidence-informed about how to promote models and methods. Evid Policy 6(2):145–159 more routine and deep evidence use in schools (i.e., they can Bremm N, Eiden S, Neumann C, Webs T, Ackeren I van, Holtappels HG (2017). tailor their activities and processes to what they have been Evidenzbasierter Schulentwicklungsansatz für Schulen in herausfordernden learning from MA educators regarding actual evidence use, Lagen. Zum Potenzial der Integration von praxisbezogener Forschung und Entwicklung am Beispiel des Projekts „Potenzialeentwickeln—Schulen Stär- supply, and demand). ken“ [Evidence-based school development approach for schools in challen- Conaway (2020), formerly research director for MA, provides ging circumstances. On the potential of integrating practice-oriented research research-grounded and practice-grounded insights regarding how and developmentusing the example of the project "Developing potential— to potentially move toward next level evidence use. Broadly, she strengthening schools”]. In: Manitius V, Dobbelstein P (eds) Schu- writes lentwicklungsarbeit in herausfordernden Lagen [School development in challenging circumstances]. Waxmann, Münster, pp. 140–158 If we want research to matter…we need to devote resources Brown C (2017) Research learning communities: how the RLC approach enables teachers to use research to improve their practice and the benefits for students to building relationships and strengthening organisational that occur as a result. Res All 1(2):387–405. https://doi.org/10.18546/RFA.01.2.14 practices, in service of building organisations that learn Brown C (2020) The networked school leader: how to improve teaching and stu- (p. 2). dent outcomes using learning networks. Emerald, London Brown C, Flood J (2019) Formalise, prioritise and mobilise: how school leaders More specifically, she highlights the importance of several secure the benefits of professional learning networks. Emerald, London specific aspects and structures/arrangements, most of which are Brown C, Schildkamp K, Hubers M (2017) Combining the best of two worlds: a evident and/or incipient in one or more of the cases we reviewed. conceptual proposal for evidence-informed school improvement. Educ Res 59 For example, she highlights the potential of learning networks (see (2):154–172 the English and Catalonia cases to learn more about their rela- Brown C, Malin J (2017) Five vital roles for school leaders in the pursuit of evidence of evidence-informed practice. Teachers College Record tively longstanding and more recent embrace of PLNs, respec- Burns T, Schuller T (2007) Evidence in education: linking research and policy. tively), embedded research directors (see MA), and research- Organisation for Economic Co-operation and Development practice partnerships (relatively prevalent in MA). Busby E (2019) We are fed up waiting: more than 5000 headteachers set to march Perhaps most fundamental, and perhaps best exemplified on Downing Street to demand more cash for schools. Available via Inde- presently in England and in RP, is the overarching goal to develop pendent. https://www.independent.co.uk/news/education/education-news/ school-funding-cuts-headteachers-downing-street-march-westminster-boris- and support educational ‘organisations that learn.’ To foster such johnson-a9020756.html. Accessed 24 Aug 2019 organisations on a broad scale, we suspect, ultimately will require Cain T (2015) Teachers’ engagement with research texts: beyond instrumental, that various conditions be simultaneously met—both philoso- conceptual or strategic use. J Educ Teach 41(5):478–492 phically and materially, and at multiple levels within complex Cain T, Brindley S, Brown C, Jones G, Riga F (2019) Bounded decision-making, educational systems such as those we have profiled herein. In teachers’ reflection, and organisational learning: how research can inform teachers and teaching. Br Educ Res J 45(5):1072–1087. Early online access other words, to truly approximate an educational ‘evidence use Camacho M (2016) Vers una cultura de la innovació: reptes i oportunitats en el utopia’ will require attention to both institutional and social marc del sistema educatiu català [Towards an innovation culture: challenges factors, as highlighted via our analyses. Additionally, we suggest, and opportunities in the catalán educational system] In: Vilalta M (ed) Reptes it will require establishing conditions in which teaching is treated de l’educació a Catalunya: 2015 [Challenges of Catalonian education: 2015]. and experienced as a full-fledged profession (see Darling- Fundació Jaume Bofill, Barcelona, pp. 105–138 Catalan Government, Departament d’Educació (2019) Documents per a l’orga- Hammond et al., 2017, for jurisdictions in which this is a rea- nització i la gestió dels centres. http://educacio.gencat.cat/documents/ lity, or nearly so). Ultimately, what we suggest should be envi- IPCNormativa/DOIGC/PEC_Avaluacio_centre.pdf. Accessed 24 Aug 2019 sioned and worked toward are coherent systems in which Chapman C (2019) Making sense of reform: where next for Scottish education? teachers and educational leaders routinely and effectively can Association of Directors of Education in Scotland: The Staff College, access and integrate research evidence with other forms of Manchester Civís M, Longás J (2015) La colaboración interinstitucional como respuesta al knowledge/knowing at the “point of use” (Nutley et al., 2019, p. desafío de la inclusión socioeducativa: Análisis de 4 experiencias de trabajo en 242), as they are making consequential educational decisions. red a nivel local en Cataluña [Interinstitutional collaboration in response to the challenge of socio-educational inclusion: analysis of 4 experiences of Data availability networking at a local level in Catalonia]. Educación XXI 18(1), 213–236 Cohen DK, Mehta JD (2017) Why reform sometimes succeeds: Understanding the All datasets analysed or generated are indicated in the paper. conditions that produce reforms that last. American Educational Research Journal 54(4):644–690 Received: 10 December 2019; Accepted: 26 August 2020; Coldwell M, Greany T, Higgins S, Brown C, Maxwell B, Stiell B, Stoll L, Willis B, Burns H (2017) Evidence-informed teaching: an evaluation of progress in England. Department for Education, London Conaway C (2020) Maximizing research use in the world we actually live in: rela- tionships, organizations, and interpretation. Educ Financ Policy 15(1):1–10 Note Darling-Hammond L (2010) The flat world and education. Teachers College Press, 1 It should be noted that in RP, as in most Länder, there is no longitudinal individual New York data at pupil level available that could show developments in a more Darling-Hammond L, Burns D, Campbell C, Goodwin AL, Hammerness K, Low differentiated way. EL, McIntyre A, Sato M, Zeichner K (2017) Empowered educators: How high-performing systems shape teaching quality around the world. John Wiley & Sons, San Francisco References Datnow A, Park V, Kennedy-Lewis B (2013) Affordances and constraints in the Act on the Education Quality Improvement (LOMCE, 2013). Ley Orgánica 8/2013, context of teacher collaboration for the purpose of data use. J Educ Adm 51 de 9 de diciembre, para la mejora de la calidad educativa. https://www.boe.es/ (3):341–362 buscar/pdf/2013/BOE-A-2013-12886-consolidado.pdf. Accessed 23 Apr 2019 Díaz Costa MA (2009) Impacto de la investigación educativa en la práctica docente. Act on the Right to Education (LODE, 1985). Ley Orgánica 8/1985. https://www. Dissertation, Universidad de Granada boe.es/eli/es/lo/1985/07/03/8/con Accessed 23 Apr 2019 Díaz-Gibson J, Civís M, Carrillo E, Cortada M(2015) El liderazgo y la gobernanza Azorín C (2019) The emergence of professional learning networks in Spain. J Prof colaborativa en proyectos educativos comunitarios [Leadership and colla- Cap Community 4(1):36–51 borative governance in community education projects] Pedogog Soc Rev Bauman Z (2012) Liquid modernity. Polity Press, Cambridge Interuniv 26:59–83 12 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 ARTICLE Dormann C, Binnewies C, Koch AR, Ackeren IV, Clausen M, Preisendörfer P, development in a managerial system—the relationship between school Zlatkin-Troitschanskaia O (2016) Transferring best evidence into practice: management and school supervision in the USA]. In: Klein ED, Bremm N Assessment of evidence-based school management. J Educ Res online 8 (eds) Unterstützung, Kooperation, Kontrolle. Zum Verhältnis von Schu- (3):14–38 laufsicht und Schulleitungen in der Schulentwicklung (Arbeitstitel). Educa- EduCaixa (2019) EduCaixa Programme. https://www.educaixa.com/es/ Accessed tional Governance. Springer VS 23 Aug 2019 Kotthoff HG, Böttcher W, Nikel J (2016) Die‚ Schulinspektion‘ zwischen Wir- EEF (2014) EEF launches £1.5 million fund to improve use of research in schools. kungshoffnungen und Wirksamkeit [The ‘school inspection’ between hopes Available via EEF. https://educationendowmentfoundation.org.uk/news/eef- of effect and effectiveness.]. In: Altrichter H, Maag Merki K (eds) Handbuch launches-15-million-fund-to-improve-use-of-research-in-schools/ Accessed Neue Steuerung im Schulsystem, 2nd edn. Springer VS, Wiesbaden, 27 Aug 2019 pp. 325–359 EEF (2018) EEF toolkit new survey shows 59% of senior leaders using it to inform Kronsfoth K, Muslic B, Graf T, Kuper H (2018) Der Zusammenhang zwischen their decisions. Available via EEF. https://educationendowmentfoundation. Führungsdimensionen in der Schulleitung und der Nutzung von Ergebnis- org.uk/news/eef-toolkit-new-survey-shows-59-of-senior-leaders-using-it-to- rückmeldungen aus Vergleichsarbeiten [The relationship between leadership inform-their-/. Accessed 27 Aug 2019 dimensions in school management and the use of feedback from comparative Ehren M (2018) Accountability and trust: two sides of the same coin? Invited talk statewide assessments]. Die Dtsch Sch 110(1):47–64 as part of the University of Oxford’s: Intelligent accountability symposium II, Kühle B, van Ackeren I (2012) Wirkungen externer Evaluationsformen für eine Oxford, 12 Dec 2018 evidenzbasierte Schul-und Unterrichtsentwicklung [Effects of external forms Farley-Ripple E, May H, Karpyn A, Tilley K, McDonough K (2018) Rethinking of evaluation for evidence-based school and teaching development]. In: connections between research and practice in education: a conceptual fra- Ratermann M, Stöbe-Blossey S (eds) Governance von Schulund Ele- mework. Educ Res 47(4):235–245 mentarbildung. VS Verlag für Sozialwissenschaften, pp. 45–62 Fend H (2009) Neue Theorie der Schule: Einführung in das Verstehen von Bil- Kuhn HJ (2014) Anspruch, Wirklichkeit und Perspektiven der Gesamtstrategie der dungssystemen. [New school theory: introduction to understanding educa- KMK zum Bildungsmonitoring. [Entitlement, reality and perspectives of the tion systems]. Springer-Verlag, Berlin overall strategy of the KMK [Standing conference of the Ministers of Edu- Gairin J (2015) Autonomy and school management in the Spanish context. J Educ cation and Cultural Affairs of the Länder in the Federal Republic of Ger- Cult Psychol Stud 11:103–117 many] on education system monitoring]. Die Dtsch Sch 106(4):414–426 Galdin-O’Shea H (2015) Leading ‘disciplined enquiries’ in schools. In C. Brown Kultusministerkonferenz (2015) Text der überarbeiteten Gesamtstrategie zum (ed) Leading the use of Research & Evidence in schools, (London, IOE Press) Bildungsmonitoring [Text of the revised overall strategy for education (pp. 91–106) monitoring]. Available via kmk. https://www.kmk.org/fileadmin/Dateien/pdf/ Godfrey D (2016) Leadership of schools as research-led organizations in the PresseUndAktuelles/2015/Gesamtstrategie_Endfassung_DOK.pdf Accessed English educational environment: cultivating a research engaged school 29 Aug 2019 culture. Educ Manag Adm Leadersh 44(2):301–321 Labaree DF (1997) Public goods, private goods: the American struggle over edu- Gough D, Tripney J, Kenny C, Buk-Berge E (2011) Evidence informed policy- cational goals. Am Educ Res J 34(1):39–81 making in education in Europe. EIPEE final project report Lai M, Wilson A, McNaughton S, Hsia S (2014) Improving achievement in sec- Graves S, Moore A (2017) How do you know what works, works for you? An ondary schools: impact of a literacy project on reading comprehension and investigation into the attitudes of school leaders to using research evidence to secondary schools qualifications. Read Res Q 49(3):305–334 inform teaching and learning in schools. School Leadersh Manag 38 Lee WO, Napier DB, Manzon M (2014) Does context still matter? The dialectics of (3):259–277 comparative education. Asia Pac J Educ Publ 34(2):139–152 Greany T (2014) Are we nearly there yet? Progress, issues and possible next steps Lubienski C (2020) The future of research use? In: Malin J, Brown C (eds) The role for a self-improving school system. IOE Press, London of knowledge brokers in education: connecting the dots between research and Greany T (2015) How can evidence inform teaching and decision making across practice. Routledge, New York, pp. 1–12 21,000 autonomous schools? Learning from the journey in England. In: Luhmann N (1970) Soziologische Aufklärung [Sociological enlightenment]. Brown C (Ed) Leading the use of research & evidence in schools. IOE Press, Soziologische Aufklärung 1. VS Verlag für Sozialwissenschaften, Wiesbaden, London, pp. 11–29 pp. 66–91 Greany T, Earley P (2018) The paradox of policy and the quest for successful Maier U, Metz K, Bohl T, Kleinknecht M, Schymal M (2011) Effekte testbasierter leadership. Prof Dev Today 19(3&4):6–12 Rechenschaftslegung auf die datenbasierte Schul-und Unterrichtsentwicklung Greany T (2017) Karmel Oration: leading schools and school systems in times of [Effects of test-based accountability on data-based school and classroom change—a paradox and a quest, Paper presented at Australian Council for development]. Z Pädagog 56(1):112–128 Educational Research (ACER) Research Conference, Melbourne, Australia, Malin JR, Brown C, Trubceac AS (2018) Going for broke: A multiple-case study of 28–29 August 2017. brokerage in education. AERA Open 4(2):1–14 Hackmann DG, Malin JR, Ahn J (2019) Data use within an education-centered Malin JR, Paralkar V (2017) Educational knowledge brokerage and mobilization: cross-sector collaboration. J Educ Adm 57(2):118–133. https://doi.org/ the Marshall Memo case. Int J Educ Policy Leadersh 12(7):1–20 10.1108/JEA-08-2018-0150 Malin JR, Brown C (2020) Joining worlds. In: Malin J, Brown C (eds) The role of Hedberg C (2018) Evidence use in Massachusetts school districts. http://www.doe. knowledge brokers in education: connecting the dots between research and mass.edu/research/reports/ Accessed 4 Oct 2019 practice. Routledge, New York, pp. 1–12 Heinrich M (2015) Zur Ambivalenz der Idee evidenzbasierter Schulentwicklung. Marchesi A, Martin E (2002) Evaluation in secondary education. snapshot from a Das Beispiel“Schulinspektion”-fortschrittlicher Rückschritt oder Innovation? controversial era. Institution IDEA, Madrid [On the ambivalence of the idea of evidence-based school development. The Martin GP, Williams O (2019) Evidence and service delivery. In: Boaz A, Davies H, example of “school inspection”—progressive regression or innovation?] Z Fraser A, Nutley S (eds) What works now? Evidence-informed policy and Pädagog 61(6):778–792 practice. Policy Press, Bristol, UK, pp. 41–57 Higher Education Funding Council, England (HEFCE) (2011) Assessment fra- Martinez X (2019) La recerca educativa, també al servei de les escoles. Models mework and guidance on submissions. HEFCE emergents per a la millora de l’impacte de la recerca educativa. In: Riera J (ed) Hood C (1998) The art of the state, culture rhetoric and public management. Reptes de l’educació a Catalunya. Anuari 2018. Fundació Jaume Bofill, Bar- Clarendon Press, Oxford celona, pp. 536–577 Horsford SD, Scott JT, Anderson GL (2018) The politics of education policy in an Mincu M (2014) Inquiry paper 6: teacher quality and school improvement—what era of inequality: possibilities for democratic schooling. Routledge, Abingdon is the role of research? In: the British Educational Research Association/The Howland G (2015) Structural reform: the experience of ten schools driving the Royal Society for the encouragement of Arts, Manufactures and Commerce development of an all-age hard federation across a market town in northern (eds) The role of research in teacher education: reviewing the evidence. England. Manag Educ 29(1):25–30 Available via bera. https://www.bera.ac.uk/wp-content/uploads/2014/02/ Ion G, Diaz A, Gairin J (2019) Developing the teachers’ professional capital BERA-RSA-Interim-Report.pdf, Accessed 8 Aug 2019 through research informed innovations. The experience of primary education Mintrop R (2015) Public management reform without managers: the case of teachers in Catalonia. Paper presented to the European educational research German public schools. Int J Educ Manag 29(6):790–795 conference, Hamburg, 3–6 September 2019 Demski D, Rosenbusch C, van Ackeren I, Clausen M & Schmidt U (2012). Ion G, Gairín J (2019). Research informed practices in Spanish schools: research Steuerung von Schule durch evidenzbasierte Einsicht? Konzeption und erste report. Unpublished document. http://pagines.uab.cat/pbe/ca Befunde des Forschungsverbundes EviS [Governing school through evidence- Kahneman D (2011) Thinking, fast and slow. Allen Lane, London based insight? Conception and first findings of the research project EviS]. In: Klein ED, Bremm N (2020) Schulentwicklung im manageriell geprägten System— Hornberg S & Parreira do Amaral M (eds) Deregulierung im Bildungssystem Das Verhältnis von Schulleitung und Schulaufsicht in den USA [School [Deregulation in the educational system]. Waxmann, Münster, pp. 131–150 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8 13 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-020-00587-8 Murillo FJ (2006) Retos de la innovación para la investigación educativa. In: overview with special consideration of the evidence-based control require- Escudero T, Correa AD (ed) Innovación e investigación educativa: algunos ment]. In: Holtappels HG (ed) Entwicklung und Qualität des Schulsystems. ámbitos relevantes. La Muralla, Madrid, pp. 23–54 Neue empirische Befunde und Entwicklungstendenzen [Development and Murillo J, Perines H (2017) Cómo los docentes no universitarios perciben la quality of the school system. New empirical findings and developmental investigación educativa. Rev complut educ 28(1):81–99 tendencies]. Waxmann, Münster, pp. 241–259 Muslic B (2017) Kopplungen und Entscheidungen in der Organisation Schule Van Geel M, Keuning T, Visscher A, Fox JP (2016) Assessing the effects of a [Interconnections and decisions in the organization school]. Springer, school-wide data-based decision-making intervention on student achieve- Wiesbaden ment growth in primary schools. Am Educ Res J 53(2):360–394 Nutley S, Davies H, Hughes J (2019) Assessing and labelling evidence. In: Boaz A, Vanlommel K, Van Gasse R, Vanhoof J, Van Petegem P (2017) Teachers’ decision- Davies H, Fraser A, Nutley S (eds) What works now? Evidence-informed making: data based or intuition driven? Int J Educ Res 83:75–86 policy and practice. Policy Press, Bristol, UK, pp. 225–246 Walker M, Nelson S, Bradshaw S, Brown C (2019) Teachers’ engagement with Oancea A (2018) The practice of educational research. International handbook of research: what do we know? A research briefing. https://www.nfer.ac.uk/ philosophy of education. Springer, Cham teachers-engagement-with-research-what-do-we-know-a-research-briefing/. OECD (2020) Education at a glance: teachers’ teaching and working time. OECD Accessed 30 Aug 2019 Education Statistics (database), https://doi.org/10.1787/d3ca76db-en. Acces- Whitty G, Wisby E (2017) Is evidence-informed practice any more feasible than sed 12 July 2020 evidence-informed policy? Presented at the British Educational Research Papay JP, Mantil A, Murnane RJ, An L, Donohue K, McDonough A (2020) Lifting Association annual conference, Sussex, 5–7 September 2017 all boats? Accomplishments and challenges from 20 years of education Willingham DT (2012) When can you trust the experts?: how to tell good science reform in Massachusetts. Educational Opportunity in MA, Brown University, from bad in education. John Wiley & Sons, Hoboken Providence Wise D (2015) In measures of economic strength, social cohesion, and fiscal Perines H (2017) Las murallas invisibles entre la investigación educativa y los position, blue and purple states are generally outperforming red states [Blog docentes. Cienc Educ 1(1):11–21 post]. LSE USCentre. Available via blogs.lse. https://blogs.lse.ac.uk/ Perines H (2018) ¿Por qué la investigación educativa no impacta en la práctica usappblog/2015/03/20/in-measures-of-economic-strength-social-cohesion- docente? Estud sobre educ 34:9–27 and-their-fiscal-position-blue-and-purple-states-are-generally- Piazza P (2017) Neo-democracy in educational policy-making: a critical case study outperforming-red-states/. Accessed 10 Dec 2019 of neoliberal reform in Massachusetts. J Educ Policy 32(2):211–233 Powell WW, DiMaggio P (1991) The neo institutional analysis in organizational analysis. University of Chicago Press, London Acknowledgements Robinson V (2017) Foreword. In: Earley P, Greany T (eds) School leadership and We acknowledge support by the Open Access Publication Fund of the University of system reform in the 21st Century. Bloomsbury, London Duisburg-Essen. Rose J, Thomas S, Zhang L, Edwards A, Augero A, Rooney P (2017) Research Learning Communities: Evaluation report and executive summary. Available via Education Competing interests Endowment Foundation. https://educationendowmentfoundation.org.uk/public/ The authors declare no competing interests. files/Projects/Evaluation_Reports/Research_Learning_Communities.pdf. Accessed 15 Aug 2019 Sahlberg P (2016) The global educational reform movement and its impact on Additional information schooling. In: Mundy K, Green A, Lingard B, Verger T (eds) The handbook Correspondence and requests for materials should be addressed to J.R.M. or I.v.A. of global education policy. John Wiley and Son, Ltd, Chichester, pp. 128–144 Secretariat of the Standing Conference of the Ministers of Education and Cultural Reprints and permission information is available at http://www.nature.com/reprints Affairs of the Länder in the Federal Republic of Germany (2019) The Edu- cation System in the Federal Republic of Germany 2016/2017. A description Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in of the responsibilities, structures and developments in education policy for published maps and institutional affiliations. the exchange of information in Europe. KMK, Bonn. retrieved from KMK. https://www.kmk.org/fileadmin/Dateien/pdf/Eurydice/Bildungswesen-engl- pdfs/dossier_en_ebook.pdf, Accessed 5 Jul 2020 Shober AF (2016) Individuality or community? Bringing assessment and Open Access This article is licensed under a Creative Commons accountability to K-16 education. In: Loss C, McGuinn P (eds) The con- Attribution 4.0 International License, which permits use, sharing, vergence of K-12 and higher education: policies and programs in a changing adaptation, distribution and reproduction in any medium or format, as long as you give era. Harvard Education Press, Cambridge, MA, pp. 67–86 appropriate credit to the original author(s) and the source, provide a link to the Creative Tseng V, Coburn C (2019) Using evidence in the US. In: Boaz A, Davies H, Fraser Commons license, and indicate if changes were made. The images or other third party A, Nutley S (eds) What works now? Evidence informed policy and practice. material in this article are included in the article’s Creative Commons license, unless Policy Press, Bristol, UK, pp. 351–368 indicated otherwise in a credit line to the material. If material is not included in the van Ackeren I, Klemm K, Kühn SM (2015) Entstehung, Struktur und Steuerung article’s Creative Commons license and your intended use is not permitted by statutory des deutschen Schulsystems. Eine Einführung. [Development, structure and regulation or exceeds the permitted use, you will need to obtain permission directly from governance of the German school system. An introduction.]. Springer Fachmedien, Wiesbaden the copyright holder. To view a copy of this license, visit http://creativecommons.org/ van Ackeren I, Demski D, Klein ED (2017) Entwicklungsprobleme Neuer licenses/by/4.0/. Steuerung imSchulsystem. Ein systematisierender Überblick unter besonderer Berücksichtigung des evidenzbasierten Steuerungsanspruchs [Development © The Author(s) 2020 problems of new control systems in the school system. A systematizing Joel R. Malin1 ✉, Chris Brown2, Georgeta Ion 3, Isabell van Ackeren4 ✉, Nina Bremm5, Ruth Luzmore2, Jane Flood2 & Gul Muhammad Rind1 1 Miami University, Oxford, OH, USA. 2Durham University, Durham, England. 3Universitat Autònoma de Barcelona, Barcelona, Spain. 4University Duisburg-Essen, Duisburg, Germany. 5Zurich University of Teacher Education, Zurich, Switzerland. ✉email: malinjr@miamioh.edu; isabell.van- ackeren@uni-due.de 14 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2020)7:99 | https://doi.org/10.1057/s41599-020-00587-8