Skip Navigation

Publications & Products

Clear Search


Search by:






Release Date 

     

Type of Product (help)

Center/Program

Search Results: (1-15 of 445 records)

 Pub Number  Title  Date
REL 2021226 Identifying Students At Risk Using Prior Performance Versus a Machine Learning Algorithm

This report provides information for administrators in local education agencies who are considering early warning systems to identify at-risk students. Districts use early warning systems to target resources to the most at-risk students and intervene before students drop out. Schools want to ensure the early warning system accurately identifies the students that need support to make the best use of available resources. The report compares the accuracy of using simple flags based on prior academic problems in school (prior performance early warning system) to an algorithm using a range of in- and out-of-school data to estimate the specific risk of each academic problem for each student in each quarter. Schools can use one or more risk-score cutoffs from the algorithm to create low- and high-risk groups. This study compares a prior performance early warning system to two risk-score cutoff options: a cutoff that identifies the same percentage of students as the prior performance early warning system, and a cutoff that identifies the 10 percent of students most at risk.

The study finds that the prior performance early warning system and the algorithm using the same-percentage risk score cutoffs are similarly accurate. Both approaches successfully identify most of the students who ultimately are chronically absent, have a low grade point average, or fail a course. In contrast, the algorithm with 10-percent cutoffs is good at targeting the students who are most likely to experience an academic problem; this approach has the advantage in predicting suspensions, which are rarer and harder to predict than the other outcomes. Both the prior performance flags and the algorithm are less accurate when predicting outcomes for students who are Black.

The findings suggest clear tradeoffs between the options. The prior performance early warning system is just as accurate as the algorithm for some purposes and is cheaper and easier to set up, but it does not provide fine-grained information that could be used to identify the students who are at greatest risk. The algorithm can distinguish degrees of risk among students, enabling a district to set cutoffs that vary depending on the prevalence of different outcomes, the harms of over-identifying versus under-identifying students at risk, and the resources available to support interventions.

9/28/2021
REL 2021118 Variations in District Strategies for Remote Learning During the COVID-19 Pandemic

In spring 2020, the COVID-19 pandemic led to an unprecedented and abrupt stoppage of in-person learning in schools across the country. State education agency leaders in Kansas, Nebraska, North Dakota, and Wyoming needed information on proposed strategies in districts’ remote learning plans to ensure continuity and better support remote learning in their states. This study used document analysis to examine proposed strategies related to infrastructure; strategies and supports for instruction; and supports for teachers, students, and parents. Findings are presented separately by district Internet connectivity level, district poverty quartile, and district locale. These findings represent variations in district remote learning plans across the four states included in the study.

The study found that proposed remote learning strategies varied considerably and were often related to district characteristics. For instance, a higher percentage of districts with higher Internet connectivity before the pandemic proposed support for home-based Internet; full student access to devices; technology support; and additional supports for teachers, students, and parents. In addition, a higher percentage of nonrural districts and high-poverty districts proposed supports for students and parents, such as one-on-one meetings between students and teachers and resources for parents on remote learning.

Although district capacity to implement remote learning has likely improved since the start of the pandemic, state education agency leaders can use the findings in this report to consider providing more support to districts with persistent Internet connectivity challenges. Leaders can also use the report to inform additional data collection to examine how remote learning strategies have evolved and to help determine the implications of the shift on student learning.

9/27/2021
REL 2021116 Factors Associated with Grade 3 Reading Outcomes of Students in the Commonwealth of the Northern Mariana Islands Public School System
Few elementary students on the Commonwealth of the Northern Mariana Islands (CNMI) are scoring at grade level or higher on the ACT Aspire reading assessment. To better understand factors associated with the reading proficiency of CNMI grade 3 students, stakeholders there asked the Regional Educational Laboratory Pacific to examine the demographic characteristics and education experiences of students who demonstrated reading proficiency by grade 3. The study focused on grade 3 students who were enrolled in CNMI public schools from 2014/15 to 2018/19. It found that female students, students who did not receive free or reduced-priced lunch, students who were older at the time of kindergarten entry, Filipino students, and students who did not change schools were more likely to demonstrate reading proficiency in grade 3 than other students. There was no difference in grade 3 reading proficiency between students who had enrolled in Head Start and students who had not.
9/21/2021
REL 2021108 Supports Associated with Teacher Retention in Michigan

Statewide teacher shortages are hindering Michigan’s efforts to ensure that all students have equitable access to qualified teachers. Implementing teacher supports—which may be policies, practices, or programs—to increase teacher retention offers a way to alleviate shortages. This study identified supports implemented by local education agencies (traditional school districts and charter schools) that are associated with teacher retention. The study examined local teacher retention rates from 2013/14 to 2018/19 and teachers' responses to a survey about teacher supports in their local agencies and their perceptions of those supports.

Average annual teacher retention rates among Michigan’s local education agencies ranged from 33 percent to 100 percent in the six-year period. The likelihood that teachers would remain teaching in their local education agency was higher in local education agencies that served lower percentages of students who were economically disadvantaged, higher percentages of students who were White, and higher percentages of students proficient in English language arts. And the likelihood was higher in agencies that had regular supportive communication between new teachers and school leaders, implemented mentoring programs, provided new teachers with an orientation to their school, allowed teachers to set goals in their evaluations, and provided teachers with sufficient instructional resources. The study also found that supports associated with teacher retention varied by the type of local education agency and the percentage of students who were economically disadvantaged. Findings from this study can help education agencies in Michigan prioritize which of 30 teacher supports examined merit more rigorous investigation.

9/20/2021
REL 2021100 Variation in Mentoring Practices and Retention across New Teacher Demographic Characteristics under a Large Urban District's New Teacher Mentoring Program
A large urban school district wanted to understand how its first-year teacher mentoring program might better support the district goals of increasing retention and maintaining a diverse workforce. This study investigated new teachers' participation in that program; how participation varied across teacher characteristics, especially how participation varied by the racial/ethnic makeup of new teacher–mentor pairs; and how participation in various aspects of the program was related to new teacher retention after the first year. The study found that over 40 percent of new teachers spent at least 10 hours a month meeting with their mentor but that more than 25 percent spent less than half that much time in mentoring meetings. There were also large differences by race/ethnicity in the proportions of new teachers who reported spending time on specific topics: White new teachers were almost twice as likely as Black new teachers to report spending substantial time on classroom management. New teacher retention was related to the amount of time new teachers spent meeting with their mentor, to whether new teachers reported spending substantial time with their mentor addressing classroom management, and to the racial/ethnic alignment of new teacher–mentor pairs.
9/16/2021
REL 2021113 Using Enhanced Coaching of Teachers to Improve Reading Achievement in Grades PreK–2 in Chicago Public Schools
Chicago Public Schools is working to improve early literacy outcomes through a multiyear professional development initiative for preK–2 teachers. The P–2 Balanced Literacy Initiative aims to improve literacy instruction by training teachers to implement effective early literacy instruction balancing systematic foundational skills instruction with reading and writing instruction involving rich, complex texts. The initiative began in 2016/17 and served 23 percent of all district elementary schools by 2018/19. The district designated 26 of the 115 elementary schools implementing the initiative in 2018/19 to receive enhanced supports, including intensive, site-based coaching, to support students’ independent reading. This study compared the reading achievement of students who attended schools that received the enhanced supports (priority schools) with the reading achievement of students who attended similar schools that received only the initiative’s standard supports (nonpriority schools). It also examined differences between priority and nonpriority schools in teachers’ and administrators’ participation in professional development sessions and looked at the successes and challenges of implementation. The study found that one year after implementation of the initiative, attending a priority school did not lead to higher end-of-year reading achievement than attending a nonpriority school after other factors were adjusted for. Teachers and administrators in priority schools were more likely than those in nonpriority schools to participate in the initiative’s core professional development sessions. Interviews with select district, network, and school leaders; instructional support coaches; and teachers suggest that several aspects of the initiative’s professional development were valuable, most notably the opportunities for teachers to deepen their understanding of the initiative’s professional development, receive feedback through observation and school-based coaching, and learn from one another. But instructional support coaches’ limited capacity, due to competing responsibilities, was a challenge. District leaders might consider increasing the number of coaches available and limiting their competing priorities so they can focus on the initiative.
9/15/2021
REL 2021099 Exploring Implementation of Attendance Supports to Reduce Chronic Absenteeism in the Providence Public School District
In recent years Rhode Island’s Providence Public School District (PPSD) has put initiatives in place to reduce high chronic absenteeism. This study explored attendance supports aimed at reducing chronic absenteeism that PPSD schools implemented in the 2018/19 school year. Although some schools had attendance supports in place before 2018/19, in 2018 the district added new requirements for schools to address chronic absenteeism. The study investigated what attendance supports were most commonly implemented with fidelity in 2018/19 by schools in which chronic absenteeism decreased between 2017/18 and 2018/19. Schools in which chronic absenteeism decreased implemented text messaging, phone calls, and mentorship programs with fidelity more frequently than schools in which chronic absenteeism increased. The study also looked more closely at one support in particular—text messaging to parents and guardians of students—to examine how implementation varied across schools. Some schools used a targeted approach for contacting parents and guardians of students who might be at risk for chronic absenteeism, translating content to reach parents and guardians in their preferred language; this could create opportunities to reach parents and guardians in ways that other attendance supports do not. Descriptive analyses showed that during the 2018/19 school year the use of attendance-related text messaging increased more quickly in schools in which chronic absenteeism decreased between 2017/18 and 2018/19 than in schools in which chronic absenteeism increased, where the use of attendance-related text messaging remained flat.
9/14/2021
REL 2021115 The Effect of Discipline Reform Plans on Exclusionary Discipline Outcomes in Minnesota
In 2017 the Minnesota Department of Human Rights identified 43 local education agencies in the state as being in violation of the Minnesota Human Rights Act for their use of exclusionary discipline practices (suspensions, exclusions, and expulsions) at higher rates for American Indian students, Black students, and students in special education, as well as for their overall use of discipline practices. The department agreed not to pursue legal action against any identified local education agency that created and implemented a plan to reform its discipline practices. This study examined the use of exclusionary discipline practices by Minnesota local education agencies from 2014/15 through 2018/19 and the extent to which the creation of discipline reform plans by identified local education agencies was associated with changes in discipline outcomes. The study found that creating a discipline reform plan was not associated with a statistically significant change in exclusionary discipline actions experienced by students.
9/13/2021
REL 2021098 Using Promotion Power to Identify the Effectiveness of Public High Schools in the District of Columbia
This study estimated the promotion power of public high schools in the District of Columbia. Promotion power is a measure of school effectiveness that distinguishes a school/s contributions to student outcomes from the contributions of the background characteristics of the students it serves. Promotion power scores are distinct from status measures such as graduation rate and college enrollment rate because they account for prior student achievement and other student background characteristics in measuring schools’ contributions. They complement value-added measures by using similar methods to examine additional, longer-term outcomes. The study found wide variation in high schools’ promotion power for college-ready SAT scores, high school graduation, and college enrollment. Schools with high promotion power for high school graduation were also more likely to have high promotion power for college enrollment. Student background characteristics were less strongly related to promotion power scores than to status measures, suggesting that high schools serving differing student populations can show strong promotion power.
9/1/2021
REL 2021093 State-Funded Preschool in the Last Frontier: Alaska's Pre-Elementary Grant Program
Created in 2016, Alaska's Pre-Elementary Grants (PEGs) allow school districts to design, develop, and expand affordable and accessible preschool in their communities. PEGs aim, in particular, to serve historically disadvantaged students. This study aimed to help Alaska stakeholders better understand how districts implemented the grants and what were the characteristics and outcomes of children who participated in PEG programs. Based on analyses of documents, interviews, and administrative data, the study found that PEG districts served a higher proportion of Alaska Native students, English learner students, and students in rural remote schools than did non-PEG districts and that these differences increased between 2016/17 and 2018/19 as more districts received funding. PEG districts used the program’s flexibility primarily to provide or support part-day preschool. In addition, students' participation in state-funded preschool between 2013/14 and 2017/18, including PEG participation, was positively related to kindergarten readiness, kindergarten and grade 2 English language proficiency, kindergarten and grade 1 attendance, and grade 3 assessment scores in math but not to grade 3 assessment scores in reading. The study findings have important implications for Alaska’s efforts to expand preschool and might also be of interest to other predominantly rural states that are considering similar efforts. Specifically, the implementation findings can help practitioners, program directors, and state agency staff members in Alaska provide more targeted support to districts and modify the grant program in future years. The promising findings on the relationship between PEG participation and student outcomes point to the need for more rigorous research on this topic—an effort that would benefit from improved data collection.
8/31/2021
REL 2021097 First-Year Effects of Early Indicator and Intervention Systems in Oregon

Although Oregon has made recent gains in its overall high school graduation rate, 21 percent of public school students entering grade 9 in 2014 did not graduate within four years, by 2018. To improve graduation rates, Oregon voters approved Ballot Measure 98 in 2016 to fund dropout prevention and college and career readiness initiatives in high schools. Many districts used the funding to adopt an early indicator and intervention system (EIIS) to identify students who are not on track to graduate on time by monitoring related indicators, such as chronic absenteeism, disciplinary infractions, course progression, and academic performance, through a frequently updated data system. Districts can tailor the system by setting their own on-track thresholds for each indicator to identify students at risk of not graduating on time, assigning those students to interventions, and monitoring student response to the interventions.

This study took advantage of the additional funding being offered to districts across the state to look at first-year effects on chronic absenteeism, disciplinary infractions, course progression, and academic performance by comparing the outcomes in 65 districts that adopted an EIIS to the outcomes in a set of similar districts that used the additional funding for other dropout prevention or college and career readiness initiatives. The study offers insight into the effectiveness of early efforts to scale up EIISs, a popular school-level intervention. EIIS adoption appears to have reduced the percentage of students who were chronically absent by 3.9 percentage points but does not appear to have had positive effects on the three other student outcomes during the first year: the percentage of students with disciplinary infractions, the percentage of grade 10 students who had acquired enough credits by the end of grade 9 to be considered on track for on-time graduation, or the percentage of grade 11 students meeting or exceeding proficiency standards on state math and English language arts tests. The findings offer the Oregon Department of Education information on the early effects of its efforts to promote EIIS across Oregon. The findings can also be used by other state and district education leaders to inform their considerations to scale up EIIS or other similar programs.

8/30/2021
REL 2021111 Professional Development Incentives for Oregon's Early Childhood Education Workforce: A Randomized Study
Many states seek to increase the education levels of their early childhood education (ECE) workforce to improve the quality of care for children. Oregon encourages all ECE workforce members to sign up for a career lattice, a career pathway system that helps them determine goals related to increasing their education. The state also offers incentives for reaching specific steps in the career lattice and scholarships for college credit and community-based training. This study used two randomized controlled trials in 2018 and 2019 to test whether sending emails and offering different financial incentives to Oregon ECE workforce members increased career lattice sign-up and increased education and training levels or workplace retention. The study found that sending emails encouraging career lattice sign-up had no detectable impact on career lattice sign-up or workplace retention. Sending emails offering a monetary incentive at an earlier-than-usual step on the career lattice had a positive impact on training hours recorded but no detectable impact on career lattice movement, college credit hours earned, or workplace retention. Sending emails about automatic enrollment in a scholarship program had no detectable impact on scholarship use, career lattice movement, college credit hours earned, or workplace retention. Lastly, after participants were randomly assigned to study groups, the email campaigns were implemented as planned, reaching all intended participants, although the interventions ended sooner than planned because of a state policy change. The findings suggest that low-touch interventions such as emails have promise for increasing training hours but are not sufficient to induce changes in career lattice sign-up, continuing postsecondary education, or workplace retention for Oregon ECE workforce members. These results have implications for future research, in addition to demonstrating how better messaging and supports can mitigate barriers to further education and training and how email campaigns can be leveraged for workforce communication efforts. This information is particularly relevant for state agencies and education and training providers.
8/24/2021
REL 2021107 Characteristics and Performance of High School Equivalency Exam Takers in New Jersey
Since 2014 the New Jersey Department of Education has offered three high school equivalency (HSE) exams for nongraduates seeking credentials: the GED, the High School Equivalency Test (HiSET), and the Test Assessing Secondary Completion (TASC). This study used data on exam takers who had been grade 8 students in a New Jersey public school between 2008/09 and 2013/14 and who had attempted at least one HSE exam in New Jersey between March 2014 and December 2018. It analyzed how the characteristics of exam takers differ across exams and from the characteristics of non–exam takers, how the performance of exam takers with similar backgrounds varies, and how a recent reduction in the passing threshold for two of the exams affected passing rates. Among all students who had been grade 8 students in a New Jersey public school during the study years, HSE exam takers completed fewer years of school, were more likely to have been eligible for the national school lunch program in grade 8, and were more likely to identify as Black or Hispanic than non–exam takers. GED takers had received higher grade 8 standardized test scores, were more likely to identify as White, and were less likely to have been eligible for the national school lunch program in grade 8 than HiSET and TASC takers. Under the New Jersey Department of Education's original passing thresholds, exam takers in the study sample were more likely to pass the HiSET and TASC than the GED on the first attempt (after grade 8 standardized test scores were controlled for). However, after the reduction in passing thresholds, the first-attempt passing rate was similar across the three exams. Under the new passing thresholds, two-thirds of GED takers and more than half of HiSET and TASC takers passed on the first attempt, and—when all exam attempts are included—three-quarters of all exam takers ever passed each exam.
8/23/2021
REL 2021114 Using a survey of social and emotional learning and school climate to inform decisionmaking

The District of Columbia Public Schools (DCPS) has prioritized efforts to support students' social and emotional learning (SEL) competencies, such as perseverance and social awareness. To measure students' SEL competencies and the school experiences that promote SEL competencies (school climate), DCPS began administering annual surveys to students, teachers, and parents in 2017/18. DCPS partnered with the Mid-Atlantic Regional Educational Laboratory to study how the district could use these surveys to improve students' outcomes. The study found the following:

  • Students' SEL competencies and school experiences are the most favorable in elementary school and the least favorable in middle school and the beginning of high school. This pattern suggests that schools might provide targeted supports before or during grades 6-10 to promote SEL competencies and school experiences when students need the most support.
  • The trajectories of students' SEL competencies and school experiences differed in different schools, to a similar degree as trajectories in academic measures like test scores. To understand why changes in SEL competencies and school experiences differ across schools, DCPS could explore differences in practices between schools with better and worse trajectories. In addition, DCPS could provide targeted support to schools with lower levels of positive change.
  • Of the SEL competencies and school experiences in DCPS's survey, self-management—how well students control their emotions, thoughts, and behavior—is most related to students' later academic outcomes. Programs or interventions that target self-management might have the most potential for improving students' outcomes compared to those that target other SEL competencies or school experiences.
  • In statistical models designed to predict students' future academic outcomes, SEL competency and school experience data add little accuracy beyond prior academic outcomes (such as achievement test scores and attendance) and demographic characteristics. Prior academic outcomes and demographic characteristics predict later outcomes with a high degree of accuracy, and they may implicitly incorporate the SEL competencies and school experiences. These findings suggest that DCPS would not need to use SEL competencies and school experiences to identify whether or not students are at risk of poor academic outcomes.
  • Student, teacher, and parent reports on SEL competencies and school experiences are positively related across schools, but they also exhibit systematic differences, suggesting that some respondent groups may not be aligned in their view of SEL competencies and school experiences. These differences may serve as a tool to help DCPS target efforts to improve communication among students, teachers, and parents.
8/3/2021
REL 2021117 Exploring the Potential Role of Staff Surveys in School Leader Evaluation

The Mid-Atlantic Regional Educational Laboratory partnered with the District of Columbia Public Schools (DCPS) to explore the potential use of teacher surveys in school leader evaluation. The DCPS evaluation system, like many others, currently consists of two components: an assessment on how well a school performs on a set of student achievement metrics (such as proficiency on standardized tests) and an assessment by a supervisor of the principal’s leadership across multiple domains. Incorporating teacher surveys could provide an additional perspective on principals’ leadership and performance. Examining data from two teacher surveys that DCPS has used (Panorama and Insight), the study found that:

  • Most of the domains and scales from the teacher surveys can meaningfully differentiate among schools.
  • School averages of (Insight) survey domains are more similar for adjacent years in which a school had the same leader than when the school leader changed, suggesting principals have effects on these measures.
  • When a school gets a new principal, (Insight) survey domain scores change more than student proficiency rates, suggesting (consistent with theory) that principals may affect teachers more immediately than they affect student outcomes.
  • All but two of the teacher survey domains and scales have small or moderate correlations with supervisors’ ratings of principals' leadership, with values ranging from 0.26 to 0.46. Teacher survey measures are most correlated with three of the domains assessed by DCPS principal supervisors: Instruction, Talent, and School Culture.

Overall, our findings suggest that it could be useful for DCPS to use elements of teacher surveys to bring in teachers’ perspectives on principals’ leadership related to instruction, talent, and school culture. Other districts may also wish to consider employing teacher surveys to gain an additional perspective on principals from staff who interact with the principal every day.

8/2/2021
   1 - 15     Next >>
Page 1  of  30