REL researchers will explore the properties of Pennsylvania’s school climate surveys with the aim of developing a summary index of school climate that could be calculated for each school. This project will provide diagnostic information that is useful to local educators in their school improvement efforts.

New Infographic on Fostering Social and Emotional Learning Competencies (1.5 MB)
Check out a new infographic shares findings from a REL Mid-Atlantic study that explains why schools or districts should measure SEL competencies, what they should measure, and how to get started measuring and developing these competencies for their students.

Infographic on Characteristics of High School Equivalency Exams (1.9 MB)
High school equivalency exams are tests which students who do not finish high school can take to demonstrate their skills and knowledge and earn a high school equivalency credential. This infographic shares findings from a REL Mid-Atlantic study that investigated characteristics of high school equivalency exams in New Jersey and the students who take them. States considering changing the tests they offer or the passing scores for these tests can use this information to understand how these exams compare.

Describing Characteristics and Outcomes of High School Equivalency Exam Takers in New Jersey (1.0 MB)
Since 2014 the New Jersey Department of Education has offered three high school equivalency (HSE) exams for nongraduates seeking credentials: the GED, the High School Equivalency Test (HiSET), and the Test Assessing Secondary Completion (TASC). This study used data on exam takers who had been grade 8 students in a New Jersey public school between 2008/09 and 2013/14 and who had attempted at least one HSE exam in New Jersey between March 2014 and December 2018. It analyzed how the characteristics of exam takers differ across exams and from the characteristics of non–exam takers, how the performance of exam takers with similar backgrounds varies, and how a recent reduction in the passing threshold for two of the exams affected passing rates. Among all students who had been grade 8 students in a New Jersey public school during the study years, HSE exam takers completed fewer years of school, were more likely to have been eligible for the national school lunch program in grade 8, and were more likely to identify as Black or Hispanic than non–exam takers. GED takers had received higher grade 8 standardized test scores, were more likely to identify as White, and were less likely to have been eligible for the national school lunch program in grade 8 than HiSET and TASC takers. Under the New Jersey Department of Education's original passing thresholds, exam takers in the study sample were more likely to pass the HiSET and TASC than the GED on the first attempt (after grade 8 standardized test scores were controlled for). However, after the reduction in passing thresholds, the first-attempt passing rate was similar across the three exams. Under the new passing thresholds, two-thirds of GED takers and more than half of HiSET and TASC takers passed on the first attempt, and—when all exam attempts are included—three-quarters of all exam takers ever passed each exam.

Using School Climate Index Data to Improve Social-Emotional Learning, Student Engagement and Support, and School Safety (611 KB)
Students, their families, and educators all desire a safe and supportive school climate. Research has demonstrated that school climate can influence student outcomes including academic achievement, school success, effective violence prevention, students' healthy development, and teacher retention. Check out this new fact sheet to learn about the recent findings in a REL project with Pennsylvania schools.

Using experiments to design report cards on school performance (770 KB)
Well-designed report cards on school performance can effectively communicate information and support parents in engaging with their children's schools. Experiments are one way to gather feedback on report card designs. Here we highlight some considerations for those interested in this approach.

This study assisted DCPS by exploring the properties and uses of data from their districtwide survey of social and emotional learning (SEL). The survey assigned a range of SEL competencies, including perseverance, self-management, self-efficacy, social awareness, rigorous expectations, and sense of belonging. The study analyzed existing administrative and survey data for students, parents, and teachers who were eligible to complete the survey. The results helped inform how DCPS can effectively use its data to reach and track progress towards its time-sensitive strategic goals.

The study's key research questions are:

1. How do SEL competencies evolve with age and grade for different types of students?
2. To what extent do SEL competencies and individual students' growth in SEL competencies vary within and across schools?
3. How do SEL measures complement other data that DCPS has access to for predicting future outcomes related to its strategic goals, such as test scores, graduation, absences, and successful re-enrollment?
4. How do survey-based SEL measures align between various types of respondents (students, parents, and teachers)?

The design of information displays like school report cards can have a substantial effect on users. The REL worked with the District of Columbia Office of the State Superintendent of Education to test the impacts of different design factors. In this experiment, participants viewed report cards that varied in the factors we investigated. Each participant viewed up to two visually distinct report card designs. Across the entire sample, we examined all possible combinations of design choices. After displaying each report card, we measured parents' comprehension of information presented in the report card and their self-reported evaluations of the designs. The findings from this study provided evidence of how different school report card designs affect usability.

NJDOE requested the REL's assistance to support and help develop the department's capacity to interpret measures of school-level performance before and after the shortening of its statewide student assessments. The REL examined school-level variation in school performance measures over time to assess the extent to which the change in assessment is related to the reliability of school-level measures.

Development of Pennsylvania Department of Education School Climate Index Summary (6 MB)
This project explores the properties of Pennsylvania Department of Education school climate surveys with the aim of developing a summary index of school climate that could be calculated for each school using the surveys. Data to develop measures of school climate were gathered from surveys of students, teachers, other staff, parents, and community members. Survey results are intended to provide Pennsylvania schools and educators with information that can be used for needs assessments, program development, and short- and long-term improvement planning. The measures of school climate resulting from surveys can also be used to compare schools in the state.

Why school accountability systems disproportionately identify middle schools' SWD subgroups for TSI (1.15 MB)
The purpose of this study was to understand why middle schools in two mid-Atlantic states were apparently disproportionately identified for targeted support and improvement (TSI) based on the performance of their students with disabilities (SWD) subgroups. The study used publicly available data to examine differences across school levels – elementary, middle, and high – and between all students and SWDs. It is relevant to compare all students to SWDs because accountability systems designate TSI schools as those with subgroups of students that perform poorly relative to all students. The study focused on two aspects of school accountability systems: (1) the number of schools in each school level in which enough SWDs took state assessments for the school to be held accountable for the academic proficiency of its SWD subgroup, and (2) the average performance on accountability indicators, by school level and subgroup. In both states, SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools' SWD subgroups were substantially more likely to meet states' minimum sample size requirements for including academic performance in accountability scores. Also, SWD subgroups across school levels consistently performed worse than all students on academic proficiency accountability indicators. Taken together, these findings suggest that middle schools' SWD subgroups are more likely than elementary or high school SWD subgroups to be identified for TSI because their accountability scores are more likely to include academic proficiency indicator scores – which tend to be substantially lower than the all students group's academic proficiency indicator scores. This research suggests that when designing school accountability systems, state education agencies may wish to consider how sample size affects estimates of subgroups' performance, and in particular how sample size exclusions may mask poor performance for small subgroups.

Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership (803 KB)
This project expanded on previous work with Maryland on measuring school climate. In this follow-on work, the REL worked with MSDE to build its internal capacity to effectively employ the newly developed measure of school climate and its associated index by analyzing data from the spring 2019 survey. REL staff worked with MSDE staff to build skills to independently validate and revise the survey, develop a school climate index, and develop school-level reports on the student and staff surveys.

The REL conducted a descriptive analysis to examine how specific aspects of the accountability system rules in two mid-Atlantic states affect the types of schools identified for support for student subgroups. The report found that SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools’ SWD subgroups were substantially more likely to meet states’ minimum sample size requirements for including academic performance in accountability scores.

Schools Targeted for Improvement: Are Small Sample Sizes Masking Poor School Performance? (479 KB)
This infographic summarizes the findings from the report Why School Accountability Systems Disproportionately Identify Middle Schools' SWD Subgroups for TSI. It highlights how specific aspects of the accountability system rules can affect the types of schools identified for support for student subgroups.

Using Chronic Absenteeism for School Accountability (798 KB)
The REL provided support to the Pennsylvania Department of Education (PDE) to develop its capacity on analytic issues related to chronic absenteeism, one of PDE’s School Quality and Student Success indicators identified in the state’s ESSA plan. This project improved PDE staff’s knowledge about how other state ESSA plans measure chronic absenteeism; about how well Pennsylvania’s measure usefully differentiates among schools; and about how potential variations in the planned measure of chronic absenteeism would alter the list of identified schools. The REL taught PDE staff to conduct their own analyses of their measures, enhancing their ability to inform potential decisions on the refinement of the state’s measures in future years.

Like other states across the country, Maryland is including additional measures of school performance, such as a measure of school climate, in its ESSA plan. The REL assisted the Maryland State Department of Education (MSDE) with selecting and refining school climate surveys for school staff, parents, and students. The REL provided coaching and technical support towards analyzing pilot survey data and interpreting survey results.

School Climate Infographic (3.15 MB)
School climate data can drive school improvement. This REL infographic discusses why school climate is important and how school climate can be measured.

The REL assisted the District of Columbia Public Schools (DCPS) in its analysis of data from a survey designed to assess SEL competencies and helped DCPS develop reliable and valid scales and indices of the resulting data. Review a recent presentation to learn more about this work.

During the workshop, the REL consolidated and disseminated evidence on best practices for communicating school performance data to parents, taxpayers, educators, and other stakeholders through online school report cards. The workshop focused on three themes: (1) content, (2) design, and (3) process. A recording and highlights of the workshop can be found here.