The TCTS project will build the capacity of SDP’s Offices of DEI, Research and Evaluation, and Academic Support to monitor the progress of the CLIF initiative as well as improve its design and implementation. The REL’s technical support will begin in June 2022 and continue through July 2023. The project will include the following activities: a review of existing DEI-related data collection instruments, production of a landscape analysis, and collaboratively determining how existing measures could be adopted or adapted to monitor CLIF implementation; co-developing and supporting SDP’s pilot of test measure(s); and co-designing survey items, conducting cognitive interviews, and leading a technical support session to refine survey items.

This TCTS project will include coaching for central office staff involved in leading and supporting the Equity Partners Fellowship. Coaching will begin in August 2022 and continue throughout the 2022/23 school year to help SDP leaders in the DEI office refine the program and strengthen the supports it will offer to future Equity Fellows, particularly around their action research projects. Coaching will also include modeling the data collection process to support ongoing data informed continuous improvement of the program.

The REL and NJDOE are collaborating to develop and facilitate a coaching series for 10 select New Jersey school districts to build their capacity to make data-informed decisions and implement research-based practices that promote increased hiring of teachers of color throughout the state.

The Accountability Community of Practice is working with NJDOE to ensure that the measures of school ELP used in its ESSA accountability are reliable and valid in the wake of COVID-related assessment disruptions and for the long term. The REL is providing support for the design of diagnostic analyses to assess potential biases in the schoolwide measures of English language proficiency (ELP) due to missing data from testing disruptions in 2020 and 2021, assistance in interpreting the findings from the proposed diagnostic analyses, providing technical specifications to impute missing data and weight existing data on ELP measures to be representative of each school’s full population of English learners, and coaching and dissemination support on how to effectively describe the proposed alternative ELP indicator weights system and the techniques used to handle missing student-level ELP data for a non-technical audience and stakeholders.

The Accountability Community of Practice is working with NJDOE to explore Bayesian methods, and to evaluate whether using Bayesian methods would improve the reliability of the accountability data, especially when applied to smaller subgroups. The project will provide training, code and other coding resources, and an educator-friendly infographic to NJDOE technical program support staff. Short-term outcomes include a better understanding of Bayesian methods and measures, distribution of an infographic on Bayesian stabilization, and improved capacity to implement refinements to the NJDOE accountability system. Medium-term outcomes include an increase in the reliability of school-performance measures used in NJ’s accountability system, and successful communication of the merits of those measures to NJ educators.

Under this project, REL Mid-Atlantic is providing DC Public Charter School Board (PCSB) with technical support as it develops a new framework of school performance measures. We are advising PCSB about issues related to the reliability, validity, and robustness of measures; and exploring implications of different approaches for combining and weighting the measures—with particular attention to PCSB’s desire to promote equity in its framework. The project team will provide this feedback to DC PCSB over the course of 3 to 4 virtual sessions during the first half of 2023 as PCSB seeks to establish its new framework of school performance measures.

This TCTS project will focus on the new system’s observation component. The project will help the Delaware Department of Education (DDOE) collect and learn from data from the two pilot years of the new system (2021/22 and 2022/23) as DDOE prepares to implement the system statewide in 2023/24. REL staff will partner with DDOE to help them collect and make sense of data on teachers’ and school leaders’ perceptions of the system’s observation component, factors supporting and limiting effective implementation of the observation component, and needed changes or additional support.

REL Mid-Atlantic will conduct a study to support Delaware Department of Education’s (DDOE’s) learning from the pilot implementation of its new teacher observation tool in preparation for statewide implementation of the tool. In the short term, the goal of the project is to better understand teachers’ ratings using the new system by learning (1) how consistently the indicators in the tool measure each of the three performance areas (the tool’s reliability); (2) the extent to which ratings on each of the nine indicators and three performance areas vary across teachers; (3) how strongly teachers’ ratings are associated with their estimated contributions to student achievement growth (the tool’s validity); and (4) whether ratings differ based on the characteristics of teachers or their students (potential bias in the ratings). Next, a second short-term goal is to use the information about reliability, validity, variation, and bias to suggest changes needed to facilitate effective implementation so that DDOE’s evaluation system can support teachers in improving their instructional practices. The study will also support DDOE’s long-term goal of implementing a teacher observation tool that reliably and validly assesses teachers’ instructional performance and effectively distinguishes performance to help teachers improve statewide by 2025-26. The ultimate goal of this project is to improve student outcomes.

The project will consist of three components that aim to make school climate data in Pennsylvania more useful and actionable. For this study, school climate refers to student and school staff perceptions of the quality of the school environment, specifically the quality of educator-student and peer relationships, support for learning and academic engagement, and the degree of safety and respect experienced within the school environment. First, REL staff will assess the psychometric properties of the elementary school climate survey, which was updated based on recommendations from analyses under the last REL contract. Second, to better understand the relationship between the COVID-19 pandemic and school climate, REL staff will assess changes in student and staff perceptions of school climate for middle and high schools before and during the pandemic. REL staff will also examine the association between the percentage of remote and hybrid instruction during the 2020–2021 school year and school climate in the 2021–2022 school year. Third, REL staff will work with PDE to develop a consistent scoring approach for the school climate reports that are generated for individual schools from elementary through high schools. REL staff will also help PDE modify its computer programs that analyze school climate data and provide score reports to schools. One of the goals of this project is to build Pennsylvania Department of Education’s capacity to support local education agencies (LEAs) and schools in using school climate data to understand needs and identify programs to improve the supportiveness of the school learning environment. The project also aims to increase the likelihood that LEAs and schools view the climate survey as useful, so REL staff will track school participation in the climate survey as a medium-term outcome.

The project team is working with Pennsylvania Department of Education (PDE)’s Office for Safe Schools to provide research-based support in building (1) a logic model to connect initiatives that foster supportive learning environments with measurable outcomes, (2) accessible materials, such as an infographic or fact sheet, on evidence-based practices for fostering supportive learning environments for students and staff and 3) a toolkit to support the Intermediate Unit (IU) social emotional wellness leads to assess, plan, and measure social-emotional initiatives and the impact towards creating more supportive learning environments and better social-emotional wellness for students and staff.The goals of this project are to (1) increase PDE staff’s understanding of how to support Intermediate Unit (IU), local education agency (LEA), and school staff in building supportive learning environments for students and staff; (2) deepen PDE staff’s and IU liaisons’ understanding of evidence-based practices and interventions that can support student and staff social-emotional wellness; and (3) build all partners’ understanding of Pennsylvania’s initiatives about supportive learning environments, including how the initiatives relate to one another and the extent to which current evidence suggests they may foster supportive learning environments. The logic models and information on evidence-based programs and practices will be important tools for PDE and other partners when making decisions about how to use resources (including identifying initiatives to add, adjust, or scale).

NJDOE would like to ensure that the measures of school performance used in its ESSA accountability system are reliable, both in the wake of COVID-related assessment disruptions and for the long term. Recognizing that measurement error could challenge reliability, the state is exploring the potential of a Bayesian stabilization approach to improve the accuracy of accountability measures. In an existing project, REL staff are providing training to NJDOE staff on how to implement Bayesian stabilization. In this complementary project, the REL team will design and test Bayesian stabilization models for a public report that will document the methods and their implications. We will apply Bayesian hierarchical modeling to historical subgroup-by-school-level data to stabilize accountability measures, with a short-term goal for NJDOE to develop a better understanding of how modifications to its accountability measures could increase reliability and accuracy.

REL Mid-Atlantic is supporting District of Columbia Public Schools (DCPS) to develop fidelity standards to assess the degree to which schools are implementing a multi-tiered system of support (MTSS). The coaching project will include co-facilitation of user experience conversations with DCPS MTSS staff, school level educators, and other community partners, a review of best practices literature on building fidelity standards, a scan of data currently collected through the MTSS system, and recommendations for implementation of the fidelity standards. The fidelity standards, co-developed by REL Mid-Atlantic and the DCPS MTSS team, will be disseminated and utilized during the 2023-2024 school year.

REL Mid-Atlantic will provide technical support to DC Public Schools (DCPS) to enhance their muti-tiered systems of support (MTSS) data reporting structures and dashboards and to identify pain points surfaced during the analysis of MTSS data. REL Mid-Atlantic and DCPS will then co-explore feasible solutions to address these challenges. To explore areas of improvement related to DCPS’s MTSS data dashboard and reporting structures, REL Mid-Atlantic and DCPS will co-facilitate up to five focus group sessions to learn about DCPS staff experiences with MTSS data reporting and use in years 1 and 2 of MTSS implementation. In subsequent working sessions, REL and DCPS will synthesize themes from the focus groups as well as co-explore and co-develop feasible solutions to improve these processes. DCPS and REL Mid-Atlantic also aim to address MTSS data analysis pain points surfaced through a series of working sessions that will explore root causes and feasible solutions for DCPS to adopt within their current data environment. Opportunities for additional capacity building activities in future phases of the partnership will also be explored.

This study will assess whether implementing the Toolkit to Support Evidence-Based Writing Instruction in Grades 2 Through 4 helps teachers use recommended, evidence-based practices for writing instruction and improves students’ writing quality. The toolkit builds on the What Works Clearinghouse practice guide, Teaching Elementary School Students to Be Effective Writers, to (1) make it accessible and engaging for busy educators, (2) create a structure for learning and applying practices throughout a school year, (3) promote collaborative learning and planning among teachers, and (4) offer tools for sustaining practices over time. The toolkit will be a one-stop shop that enables schools and educators to access all supports in one place, complemented by diagnostic tools to assess practices and resources for school leaders to institutionalize practices over time. This study is critical for understanding how the toolkit can support teachers and students.

This project will provide technical support (TS) to the Pennsylvania Department of Education’s Office of School Safety (OSS) to build a clear and evidence-based statewide strategy to improve the reporting and utility of Safe Schools data to improve school climate and student and staff wellness. The REL will support OSS to (1) align on a shared definition of “school safety” (2) assess the accessibility and availability of Safe Schools data, (3) identify approaches to improve the accuracy and efficiency of collecting and reporting Safe Schools data, and (4) develop recommendations for data elements and reporting formats to use in school and district reports to best support decisions about policies and programs to improve school climate and student and staff wellness.