This guide identifies practices that can improve the performance of chronically low-performing schools—a process commonly referred to as creating "turnaround schools."

The New York City P-TECH Grades 9-14 schools represent an education model that ties together the secondary, higher education, and workforce systems as a way to improve outcomes in both domains. The distinguishing feature of the P-TECH 9-14 model, as it is referred to in this report, is a partnership between a high school, a local community college, and one or more employer partners that focuses on preparing students for both college and careers -- not one or the other -- within a six-year timeframe. Education and workforce development are traditionally seen as separate spheres of influence with multiple transition points that students have been left to navigate largely on their own (for example, high school to postsecondary, and postsecondary to the workforce). P-TECH 9-14 is designed to seamlessly assist student navigation of those points -- supporting student success and mitigating the potential for students to fall through the cracks. P-TECH 9-14 schools collaborate with local colleges to provide students with an opportunity to earn a high school diploma (within four years) followed by a cost-free, industry-recognized associate's degree. During the six-year program, employer partners support P-TECH 9-14 schools by providing students with work-based learning experiences such as internships, mentoring, and job shadowing. By design, the P-TECH 9-14 model offers students the opportunity to participate in focused and accelerated high school pathways, early college, and career-focused activities. This study offers initial impact and implementation findings from the first rigorous evaluation of the model, evaluating the first seven P-TECH 9-14 schools that opened in New York City. The study leverages the random lottery process created by the New York City High School Admissions System to identify impacts. The majority of the students in the sample who participated in the admissions lotteries were academically below proficiency in both math and English language arts (ELA) prior to entering high school. [This report was written with Fernando Medina. For the executive summary, see ED605313.]

The New York City P-TECH Grades 9-14 schools represent an education model that ties together the secondary, higher education, and workforce systems as a way to improve outcomes in both domains. The distinguishing feature of the P-TECH 9-14 model, as it is referred to in this report, is a partnership between a high school, a local community college, and one or more employer partners that focuses on preparing students for both college and careers -- not one or the other -- within a six-year timeframe. Education and workforce development are traditionally seen as separate spheres of influence with multiple transition points that students have been left to navigate largely on their own (for example, high school to postsecondary, and postsecondary to the workforce). P-TECH 9-14 is designed to seamlessly assist student navigation of those points -- supporting student success and mitigating the potential for students to fall through the cracks. P-TECH 9-14 schools collaborate with local colleges to provide students with an opportunity to earn a high school diploma (within four years) followed by a cost-free, industry-recognized associate's degree. During the six-year program, employer partners support P-TECH 9-14 schools by providing students with work-based learning experiences such as internships, mentoring, and job shadowing. By design, the P-TECH 9-14 model offers students the opportunity to participate in focused and accelerated high school pathways, early college, and career-focused activities. This study offers initial impact and implementation findings from the first rigorous evaluation of the model, evaluating the first seven P-TECH 9-14 schools that opened in New York City. The study leverages the random lottery process created by the New York City High School Admissions System to identify impacts. The majority of the students in the sample who participated in the admissions lotteries were academically below proficiency in both math and English language arts (ELA) prior to entering high school. [This report was written with Fernando Medina. For the executive summary, see ED605313.]

Early colleges are a new model of schooling in which the high school and college experiences are merged, shortening the total amount of time a student spends in school. This study uses a lottery-based experimental design to examine the impact of the model on longer term outcomes, including attainment of a postsecondary credential and academic performance in 4-year institutions. Results show that a significantly higher proportion of early college students were attaining postsecondary credentials. The results also show that early college students were completing their degrees more rapidly but that their performance in 4-year institutions was still comparable with the control students. [For the corresponding grantee submission, see ED604350.]

Early colleges are a new model of schooling in which the high school and college experiences are merged, shortening the total amount of time a student spends in school. This study uses a lottery-based experimental design to examine the impact of the model on longer term outcomes, including attainment of a postsecondary credential and academic performance in 4-year institutions. Results show that a significantly higher proportion of early college students were attaining postsecondary credentials. The results also show that early college students were completing their degrees more rapidly but that their performance in 4-year institutions was still comparable with the control students. [For the corresponding grantee submission, see ED604350.]

The purpose of this study was to test the efficacy of a comprehensive program model originally developed by the National Professional Development Center on Autism Spectrum Disorder (NPDC). Sixty elementary schools with 486 participants were randomly assigned to an NPDC and services as usual condition (SAU). Significantly greater changes in program quality occurred in the inclusive NPDC programs as compared with the SAU schools. Teachers in NPDC schools reported using more evidence-based practices (EBPs) and implemented EBPs with significantly greater fidelity than teachers in SAU schools. Autistic students in NPDC schools had significantly higher total attainment of educational goals than students in SAU schools, and the two groups made equivalent progress on standardized assessment outcomes across the school year. [This is the online first version of an article published in "Journal of Autism and Developmental Disorders."]

Most students who begin at a community college leave without earning a degree. Given the growing emphasis on student success, many colleges have implemented re-enrollment campaigns designed to foster re-engagement and degree completion among former students. However, there is a lack of causal evidence on their effectiveness. We implement a text message-based re-enrollment campaign in partnership with several Florida community colleges. Former students who were previously successful academically are randomly assigned to one of two treatment groups that either receives information to simplify the re-enrollment process or receives both information and a one-course tuition waiver. When comparing outcomes of former students who received information on re-enrollment to members in the control group, we find that providing information that simplifies the re-enrollment process has a small, statistically insignificant effect on re-enrolling. In contrast, offering both information and a one-course tuition waiver to recent dropouts significantly increases the likelihood of re-enrollment by 1.5 percentage points (21 percent) and full-time re-enrollment by 0.6 percentage points (22 percent). The effects are concentrated among former students who have accumulated the most credits and those with lower grade point averages. This study highlights the importance of targeted interventions that address informational and financial barriers facing former students.

The nation's community colleges play a central role in producing a more educated workforce and promoting social mobility. They serve about 40 percent of all college students and, not surprisingly, they serve a disproportionate number of low-income and underrepresented students. But most students who enter these colleges do not graduate--only about a third of entering students earn a degree or certificate within six years. Among the many programs that have attempted to increase graduation rates, one program stands out. Developed by the City University of New York (CUNY), the Accelerated Study in Associate Programs (ASAP) is a comprehensive program that provides students with up to three years of financial and academic support and other support services. Along with those services and other forms of support comes an obligation to attend full time and participate in essential program services. An experimental evaluation of CUNY ASAP found that the program nearly doubled graduation rates after three years. This report presents findings through three years from a replication of the ASAP model at three community colleges in Ohio. Low-income students were randomly assigned either to a program group, who could participate in their colleges' new programs based closely on ASAP (called the Ohio Programs), or to a control group, who could receive the usual college services. Comparing the two groups' outcomes provides an estimate of the Ohio Programs' effects. The findings from the evaluation add to a body of evidence showing that comprehensive programs that offer academic, financial, and other forms of student support for multiple years are an effective way to increase college completion rates. The fact that the earlier CUNY ASAP findings have now been replicated in Ohio suggests that ASAP in particular can serve as a national model to help students succeed.

The recent wave of test-based accountability reforms has negatively impacted the provision of K-12 arts educational experiences. Advocates contend that, in addition to providing intrinsic benefits, the arts can positively influence academic and social development. However, the empirical evidence to support such claims is limited. We conducted a randomized controlled trial with 10,548 3rd-8th grade students who were enrolled in 42 schools that were assigned by lottery to receive substantial influxes of arts education experiences provided through school-community partnerships with local arts organizations, cultural institutions, and teaching-artists. We find that these increases in arts educational experiences significantly reduce the proportion of students receiving disciplinary infractions by 3.6 percentage points, improve STAAR writing achievement by 0.13 of a standard deviation, and increase students' compassion for others by 0.08 of a standard deviation. For students in elementary schools, which comprise 86 percent of the sample, we find that these arts educational experiences also significantly improve school engagement, college aspirations, and arts-facilitated empathy. These findings provide strong evidence that arts educational experiences can produce significant positive impacts on student academic and social development. Policymakers should consider these multifaceted educational benefits when assessing the role and value of the arts in K-12 schools.

This is the final report of a large-scale independent evaluation of the Building Assets and Reducing Risks (BARR) model in ninth grade in eleven high schools in Maine, California, Minnesota, Kentucky, and Texas. This sample of schools included large and small schools in urban, suburban, and rural areas, serving students from a wide range of demographic and socio-economic backgrounds. Funded with a validation grant from the U.S. Department of Education's Investing in Innovation (i3) program and carried out by researchers at the American Institutes for Research (AIR), this evaluation used random assignment of ninth-grade students to BARR and control conditions to estimate the impacts of the BARR model after one year. The evaluation also assessed the fidelity of implementation of BARR in the eleven study schools and identified barriers to and facilitators of successful implementation. The evaluation focused on several teacher- and student-level outcomes. The teacher outcomes included measures of teacher collaboration, and use of data, among others. The academic outcomes included course failure, students' grade point average (GPA), and performance on the Northwest Evaluation Association's (NWEA) Measures of Academic Progress (MAP) standardized reading and mathematics assessments. Student-reported experiences included measures of supportive relationships, perceptions of teachers' expectations of them, student engagement, and others. In addition to these outcomes, the report includes impact estimates for attendance, suspensions, and persistence into 10th grade. [This report was written with Brenna O'Brien, Cheryl Graczewski, So Jung Park, Feng Liu, Ethan Adelman-Sil, Lynn Hu.]

Data-based problem solving is a hallmark of research-supported practices such as positive behavioral interventions and supports. In this study, we provided members of positive behavioral interventions and supports (PBIS) teams from 38 elementary schools with professional development focused on a research-supported problem-solving model (Team-Initiated Problem Solving). We used direct observations to document procedures, practices, and outcomes before and after participating in the professional development workshop. Within the context of a randomized waitlist controlled trial, team members in the Immediate Group demonstrated greater improvement in (a) problem-solving procedures, (b) decision-making practices, and (c) meeting outcomes than did members of PBIS teams in the Waitlist Group. Our findings extend what is known about team-based problem solving and provide a framework for future research and improved practice related to decision making by school teams.

This study reports the results of a cluster RCT evaluating the impact of Enhanced Core Reading Instruction on reading achievement of grade 1 at-risk readers. Forty-four elementary schools, blocked by district, were randomly assigned to condition. In both conditions, at-risk readers received 90 minutes of whole-group instruction (Tier 1) plus an additional 30 minutes of daily, small-group intervention (Tier 2). In the treatment condition, Tier 1 instruction included enhancements to the core program and Tier 2 intervention was highly aligned with the core program. In the comparison condition, Tier 1 instruction used the same core program as treatment schools in the district and Tier 2 intervention followed standard district protocol. Significant treatment effects were found on measures of phonemic decoding and oral reading fluency from fall to winter and word reading from fall to spring. Student- and classroom-level variables predicted student response to instruction differentially by condition.

Using a cluster randomization design, schools were randomly assigned to implement Success for All, a comprehensive reading reform model, or control methods. This article reports final literacy outcomes for a 3-year longitudinal sample of children who participated in the treatment or control condition from kindergarten through second grade and a combined longitudinal and in-mover student sample, both of which were nested within 35 schools. Hierarchical linear model analyses of all three outcomes for both samples revealed statistically significant school-level effects of treatment assignment as large as one third of a standard deviation. The results correspond with the Success for All program theory, which emphasizes both comprehensive school-level reform and targeted student-level achievement effects through a multi-year sequencing of literacy instruction. (Contains 5 tables.)

The College and Career Readiness Expansion project was a five-year, federally funded effort to implement early college strategies in comprehensive high schools. This report presents the results from a quasi-experimental impact study of the project and a mixed methods examination of its implementation. Results showed that the program increased the percentage of students on-track for college in 9th and 10th grade and expanded the number of students taking dual enrollment courses, although there was no positive impact on college credits earned in high school. The implementation study found that schools were created a more future-oriented culture.

Purpose: The purpose of this two-year study was to assess the impact of the CORE program, a model that integrates technology and active learning modules in high schools by providing multi-disciplinary teams of teachers and administrators with professional development and resources to support the development of students' non-cognitive skills and increase their college and career readiness. A fidelity of implementation study was also conducted to assess the seven key program components were being implemented as intended. Methods: This Randomized Control Trial (RCT) study followed a cohort of 9th and 10th grade high school students in 28 treatment and control schools; students completed the CWRA+ assessment and non-cognitive skills, engagement, and self-efficacy scales at three timepoints. Using the hierarchical linear model (HLM), the study assessed one-year program impact and two-year program impact. The two-year program impact model suffered a high attrition (due to COVID19) and the study became a quasi-experimental design (QED) study after propensity score matching analysis was applied. Results: Findings highlight how students at CORE schools showed increased scores across the two-year program intervention. Specifically, standardized effects on CWRA+ scores, the non-cognitive skills scale, engagement scale, and efficacy scale were respectively, 0.22, 0.22, 0.23, and 0.32. The effect size for the efficacy scale (0.32) was large enough to be considered important. To selectively mention exploratory findings, the level of program exposure both in terms of whether students were enrolled in the program-trained teachers' courses and whether teachers participated in PD activities seemed correlated with a higher growth in student's CWRA+ scores. Another set of exploratory findings implied that the program impact may interact with demographic characteristics of students. Findings demonstrate a need for further testing of differences between student subgroups based on demographics, as well as the importance of buy-in from program implementers to provide the customized PD support that educators and partners at rural schools need to more effectively serve their students. The exploratory findings suggested that the program exposure of teachers and students may be a key to enhance the CORE program impact. [This report was submitted by the ICF Evaluation Team.]

In 2013, the NYC Leadership Academy (NYCLA) developed a leadership intervention--the Targeted Intensive School Support (TISS) program--in collaboration with the New York City Department of Education (NYC DOE) to support schools that were facing particular challenges. NYCLA asked the RAND Corporation to provide an independent evaluation of the program's implementation and effects, and those findings are detailed in this report. The TISS program consisted of five key components: (1) teaming and collaborative training in aligned preservice preparation programs for a principal and assistant principal (AP); (2) coplacement of a principal and AP into an NYC DOE school; (3) team-based coaching to support the principal and AP; (4) 328 hours of extended coaching over the first three years after placement; and (5) use of a diagnostic process to guide goal setting and coaching according to school needs. Implementation findings suggest that only two of the five key components were implemented with fidelity. A propensity weighting approach was used to compare schools with TISS principals to other schools with new principals trained through residency-based preservice programs who did not participate in TISS. Findings suggest no statistically significant differences between TISS schools and comparison schools on measures of student achievement, school culture, and principal retention. TISS schools underperformed relative to comparison schools on the measure of chronic student absenteeism. [Sponsored by NYC Leadership Academy through a subgrant from the U.S. Department of Education.]

With the launch of the New York City Community Schools Initiative (NYC-CS) in 2014, the New York City Department of Education (NYCDOE) has increased its focus on the implementation of a holistic strategy of education reform to address the social consequences of poverty as a means to improving student outcomes. NYC-CS is a strategy to organize resources in schools and share leadership among stakeholders so that academics, health and wellness, youth development, and family engagement are integrated into the fabric of each school. New York City is implementing this strategy at a scale unmatched nationally. In this study, the authors assessed the impact of the NYC-CS through the 2017-2018 school year. The authors assessed the effects along seven outcome domains and explored the extent to which there is heterogeneity in programmatic impact based on student- and school-level characteristics. The authors leveraged innovative quasi-experimental methodology to determine whether students in the community schools are performing better than they would be had their schools not been designated as Community Schools. The findings of this report will contribute to the emerging evidence base on the efficacy of the community school strategy and will be useful for other school district- and state-level policymakers interested in developing or refining similar interventions that support students' and communities' academic, social, and emotional well-being. [The research described in this report was prepared for the New York City Mayor's Office for Economic Opportunity (NYC Opportunity).]

Access to middle class jobs increasingly requires a college degree or credential. Individuals with postsecondary education have competitive advantage in the labor market: even when a job does not explicitly require a degree, a candidate with a degree will tend to be hired over an equally qualified candidate without one. Low-income students, in particular, along with first-generation students and racial/ethnic minorities, are underrepresented in postsecondary education Massachusetts is faced with an aging workforce where nearly half of the labor market is 45 or older. In Boston, the six-year college graduation rate for the city's 2011 public high school graduates who enrolled in college was 52 percent. This rate improves upon the 39 percent seven-year rate for 2000 graduates, yet likely is not sufficient to meet the predicted demand for a college-educated workforce. One-on-one coaching from experienced counselors when students are completing their senior year in high school and beginning college can help them succeed. A core strategy of the city-wide "Success Boston" initiative is one-on-one transition coaching. The coaching model is implemented across a network of nonprofit organizations (that provide coaching) in partnership with institutions of higher education. Transition coaching offers students sustained, proactive, and responsive support in their first two years after high school. This report refers to the transition coaching program as Success Boston Coaching (SBC). The report is designed to answer four main research questions about implementation and impact: (1) What is the effect of SBC on student success in college?; (2) How, if at all, do the impacts of SBC vary by student characteristics and features of the coaching?; (3) How is SBC implemented across partner organizations and partner colleges? and (4) What resources are necessary to implement SBC? This report focuses on the effectiveness of coaching on students' persistence and achievement early in college, answering the first and second research questions. [For the previous report, "Success Boston: Coaching for Completion. 2015-16 Implementation Report," see ED582088. For the companion report, "The Story of Scaling Up: Highlights from the Interim Report on the Impact of Success Boston's Coaching for Completion," see ED602749.]

A growing body of research points to the ways in which principals influence teachers, classrooms, and, ultimately, student achievement. New Leaders aims to prepare transformational school leaders by partnering with districts and charter schools to offer rigorous, research-based training for aspiring principals. The Aspiring Principals program is New Leaders' signature program and has three core features: selective recruitment and admission, training and endorsement, and support for principals early in their tenure. This report is a follow-up to the 2014 evaluation of New Leaders' Aspiring Principals program. Focusing on the revised program, which was first implemented in 2012, the authors present evidence of the effectiveness of the revised Aspiring Principals program and share lessons that can inform principal-preparation policy and practice. To assess the effect of New Leaders' Aspiring Principals program, researchers analyzed whether schools and students led by graduates of the program outperformed comparison schools and students in the same district, focusing on student achievement and principal retention. They also examined program graduate placement and satisfaction with the Aspiring Principals program. [For the appendixes, see ED605724.]

Across the country, school districts, their stakeholders, and policymakers have become increasingly concerned about suspensions, particularly about suspending students from elementary school and disproportionately suspending ethnic/racial minority students. Suspended students are less likely to graduate, possibly because they miss the instructional time they need to advance academically. Restorative practices have gained buy-in in the education community as a strategy to reduce suspension rates. Proactively improving relationships among students and staff and building a sense of community in classrooms and schools may make students less inclined to misbehave. And addressing severe misbehavior through a restorative approach may help students realize the impacts of their actions and make them less likely to offend again. This study of the implementation of restorative practices in the Pittsburgh Public Schools district (PPS) in school years 2015-16 and 2016-17 represents one of the first randomized controlled trials of the effects of restorative practices on classroom and school climates and suspension rates. The authors examined a specific restorative practices program -- the International Institute for Restorative Practices' SaferSanerSchools™ Whole-School Change program -- implemented in a selected group of PPS schools under a program called Pursuing Equitable and Restorative Communities, or PERC. The researchers found that PERC achieved several positive effects, including an improvement in overall school climates (as rated by teachers), a reduction in overall suspension rates, and a reduction in the disparities in suspension rates between African American and white students and between low- and higher-income students. Key Findings: Effects of the Pursuing Equitable and Restorative Communities (PERC) program in Pittsburgh Public Schools: (1) Implementation of restorative practices through PERC improved overall school climates, as rated by teachers; (2) Implementation of restorative practices reduced the average suspension rate: During the study period, average suspension rates decreased in both PERC and non-PERC schools, but rates decreased more in PERC schools; (3) Suspension rates of African American students and of those from low-income families also went down in PERC schools, shrinking the disparities in suspension rates between African American and white students and between low- and higher-income students; (4) Academic outcomes did not improve in PERC schools, and actually worsened for grades 6-8; and (5) Arrest rates among PERC schools did not decrease. Recommendations: (1) Given the constraints on teachers' time, emphasize restorative practices that can be woven into the school day; (2) Ensure that school leaders understand and can model restorative practices, including by providing mandatory professional development, books and other materials, and coaching on restorative practices; (3) Establish a mechanism for school staff to meet at least once per month as a professional learning community on restorative practices; (4) Ensure that leaders at the district level can coordinate this work; (5) Set, and update, clear expectations regarding the use of restorative practices; and (6) Implement data collection systems to collect accurate information on all types of behavioral incidents and remedies.

Since their inception in 1992, the number of charter schools has grown to more than 6,800 nationally, serving nearly three million students. Various studies have examined charter schools' impacts on test scores, and a few have begun to examine longer-term outcomes including graduation and college attendance. This paper is the first to estimate charter schools' effects on earnings in adulthood, alongside effects on educational attainment. Using data from Florida, we first confirm previous research (Booker et al., 2011) that students attending charter high schools are more likely to graduate from high school and enroll in college. We then examine two longer-term outcomes not previously studied in research on charter schools--college persistence and earnings. We find that students attending charter high schools are more likely to persist in college, and that in their mid-20s they experience higher earnings.

Single Stop U.S.A.'s Community College Initiative was designed to improve the well-being of low-income communities by connecting individuals to public benefits and other institutional and community resources to address nonacademic barriers to college completion. Through offices located on community college campuses, Single Stop provides students with a range of free services, including screenings and applications for public benefit programs; tax services, financial counseling, and legal services; and case management with referrals to a wide variety of resources and support programs across the institution and community. This report presents an evaluation of the Single Stop program and its impact on students' postsecondary outcomes. The authors examined the Single Stop program at four community college systems: Bunker Hill Community College, City University of New York, Delgado Community College, and Miami Dade College. The analysis indicates that use of Single Stop was associated with improved postsecondary outcomes. The findings suggest that access to alternative financial resources from government benefit programs alongside a network of institutional and community support programs can offer valuable support to college students.

We assessed the effects of a whole-school equity intervention implemented within a school-wide positive behavioral interventions and supports (PBIS) framework on racial inequities in school discipline in eight elementary schools with inequitable referrals for Black students. The intervention involved assessing patterns of racial disparities in school discipline decisions and providing professional development on adapting school-wide behavior systems to improve cultural responsiveness through concrete strategies targeting the patterns. After consent and matching on existing levels of racial inequities, half of the schools were randomly assigned to receive the intervention. Analyses showed that schools receiving the intervention had significant decreases in racial disparities in school discipline and rates of office discipline referrals (ODRs) for Black students, while control schools had minimal change. Results are discussed in terms of improving equity in school discipline within multitiered systems of support.

The STEM Early College Expansion Project is an effort to integrate STEM strategies with the early college model and implement this in comprehensive high schools. This report summarizes findings from two separate quasi-experimental impact studies of the model in Michigan and Connecticut. Results from Michigan showed statistically significant impacts on enrollment in college-level courses and on attainment of college credits. Treatment schools in Michigan also had descriptively lower dropout rates. The Connecticut impact study had challenges with the study design that resulted in an inability to make clear causal claims about the impact.

Chicago Public Schools initiated the Excellence in Teaching Project, a teacher evaluation program designed to increase student learning by improving classroom instruction through structured principal-teacher dialogue. The pilot began in forty-four elementary schools in 2008-09 (cohort 1) and scaled up to include an additional forty-eight elementary schools in 2009-10 (cohort 2). Leveraging the experimental design of the roll-out, cohort 1 schools performed better in reading and math than cohort 2 schools at the end of the first year, though the math effects are not statistically significant. We find the initial improvement for cohort 1 schools remains even after cohort 2 schools adopted the program. Moreover, the pilot differentially impacted schools with different characteristics. Higher-achieving and lower-poverty schools were the primary beneficiaries, suggesting the intervention was most successful in more advantaged schools. These findings are relevant for policy makers and school leaders who are implementing evaluation systems that incorporate classroom observations.

The Open Learning Initiative (OLI) is an open educational resources project at Carnegie Mellon University that began in 2002 with a grant from The William and Flora Hewlett Foundation. OLI creates web-based courses that are designed so that students can learn effectively without an instructor. In addition, the courses are often used by instructors to support and complement face-to-face classroom instruction. Our evaluation efforts have investigated OLI courses' effectiveness in both of these instructional modes--stand-alone and hybrid. This report documents several learning effectiveness studies that were focused on the OLI-Statistics course and conducted during Fall 2005, Spring 2006, and Spring 2007. During the Fall 2005 and Spring 2006 studies, we collected empirical data about the instructional effectiveness of the OLI-Statistics course in stand-alone mode, as compared to traditional instruction. In both of these studies, in-class exam scores showed no significant difference between students in the stand-alone OLI-Statistics course and students in the traditional instructor-led course. In contrast, during the Spring 2007 study, we explored an accelerated learning hypothesis, namely, that learners using the OLI course in hybrid mode will learn the same amount of material in a significantly shorter period of time with equal learning gains, as compared to students in traditional instruction. In this study, results showed that OLI-Statistics students learned a full semester's worth of material in half as much time and performed as well or better than students learning from traditional instruction over a full semester. (Contains 11 figures and 3 footnotes.)

We provide theory and evidence about how the design of college financial aid programs affects a variety of high school, college, and life outcomes. The evidence comes from an eight-year randomized trial where 2,587 high school ninth graders received a $12,000 merit-based grant offer. During high school, the program increased their college expectations and non-merit effort but had no effect on merit-related effort (e.g., GPA). After high school, the program increased graduation from two-year colleges only, apparently because of the free college design/framing in only that sector. But we see no effects on incarceration or teen pregnancy. Overall, the results suggest that free college affects student outcomes in ways similar to what advocates of free college suggest and making aid commitments early, well before college starts, increases some forms of high school effort. But we see no evidence that merit requirements are effective. Both the standard human capital model and behavioral economics are required to explain these results.

Early colleges are a new model of schooling in which the high school and college experiences are merged, shortening the total amount of time a student spends in school. This study uses a lottery-based experimental design to examine the impact of the model on longer term outcomes, including attainment of a postsecondary credential and academic performance in 4-year institutions. Results show that a significantly higher proportion of early college students were attaining postsecondary credentials. The results also show that early college students were completing their degrees more rapidly but that their performance in 4-year institutions was still comparable with the control students. [For the corresponding grantee submission, see ED604350.]

This i3 scale-up study examined the implementation and effectiveness of a 3-year literacy intervention developed by the Children's Literacy Initiative (CLI). The study was a school-level cluster randomized controlled trial conducted in 55 elementary schools from four states. Implementation results showed a high level of fidelity of intervention implementation across the treatment schools in the four urban school districts that had not previously worked with CLI. Intent-to-treat analyses of observation data showed that the intervention's effects on classroom literacy environment and teacher practices were not statistically significant in Year 1. By Year 3, however, treatment teachers who had taught in intervention schools for all 3 years were rated significantly higher in both the quality of classroom environment and the quality of literacy instructional practices than their peers in control schools. Analyses of student achievement data indicated that the CLI intervention had no statistically significant impact on students' reading achievement in any of the 3 intervention years. This held true for the overall student sample and for grade-specific subsamples, based on the overall literacy test score and for literacy subtest scores. In addition, differential impact analyses revealed that the impact of the CLI intervention on students' literacy achievement did not differ significantly by the level of students' baseline achievement. Further evidence from exploratory analyses showed that the intervention had no statistically significant impact on the English proficiency of English learner students or on teachers' knowledge of beginning reading practices. Because of the high amount of teacher attrition, an additional exploratory analysis that restricted the sample to only students of teachers who were stable across all 3 years in both conditions. There were positive and statistically significant effects for students in the CLI intervention condition who had stable teachers (teachers who had greater opportunity for intervention exposure) and who entered in Grade 1 and progressed to Grade 3 during the course of the study.

Purpose: In January 2015, the Northwest Board of Cooperative Educational Services (NW BOCES) received a five-year Investing in Innovation (i3) grant to develop and implement the System for Educator Effectiveness Development (SEED) program--an innovative professional development (PD) system designed to provide geographically isolated educators an impactful tool to improve teacher effectiveness. The purpose of this report is to document the implementation and impact evaluation of the grant on educator and student outcomes. Methods: Twenty-one schools in rural areas, where obtaining PD can be a challenge, participated in SEED between 2015 and 2020. One school closed during the study period but the majority of its students transferred to other participating schools. Because SEED was a school-level intervention, the target population consisted of all principals, assistant principals, teachers, and students in the participating sites. Correlational designs were implemented to examine the relationships between teacher-level and school-level SEED participation and the outcomes of interest, such as (1) principal engagement in teacher professional growth, (2) teacher access to and use of evidence-based and up-to-date practices, (3) teacher implementation of practice learned from PD in the classroom. A quasi-experimental design (QED) using propensity score matching (PSM) method was employed to examine difference in student achievement outcomes between the SEED schools and the matched non-SEED schools. Results: Correlational findings suggest that teachers who participated in SEED reported better outcomes compared to teachers who did not participate in SEED. Schools with high level of SEED participation had better outcomes compared to schools with low level of SEED participation. Findings of QED PSM design revealed that SEED had a statistically significant positive effect on one outcome measure (minority students' English language acquisition, or ELA) and marginally significant positive effects on four outcome measures (all related to English language acquisition and math outcomes among racial/ethnic minority and free and reduced-lunch, or FRL, students). Implications: Evaluation findings provided some positive and promising pieces of evidence to support SEED efficacy. Implementation findings suggest that the program was largely implemented with fidelity; yet, there are opportunities for improvement. Future research may consider replicating the program design and identify strategies to increase teacher participation at school level. Additional Materials: The following are appended: (1) SEED Evaluation: Technical Report; (2) Assessment of Fidelity of Implementation; and (3) Baseline Equivalence and Impact Analysis Outputs and Results.

Science education has experienced a significant transition over the last decade, catalyzed by a re-envisioning of what students should know and be able to do in science. That re-envisioning culminated in the release of the Next Generation Science Standards (NGSS) in 2013. The new standards set off a chain reaction of standards adoption and implementation across states, districts, and schools, including steps taken toward transforming science professional learning, instruction, curriculum, and assessment. It was in this dynamic context that Empirical Education conducted an impact evaluation, as part of an Investing in Innovation (i3) grant, of WestEd's Making Sense of SCIENCE project, a teacher professional learning model aimed at raising students' science achievement through improving science instruction. Under this grant, WestEd and Empirical Education also partnered with Heller Research Associates (HRA) to conduct an implementation study and a scale-up study of Making Sense of SCIENCE. The impact evaluation, which is the focus of this report, was a two-year cluster-randomized control trial (RCT) that took place in California and Wisconsin across seven school districts and 66 elementary schools in the 2016-17 and 2017-18 school years. The study randomized schools to either receive the Making Sense of SCIENCE professional learning or to the business-as-usual ("control") group, which received the professional learning (delayed-treatment) after the study ended. The study found that Making Sense of SCIENCE had a positive impact on teacher content knowledge (effect size = 0.56, p = 0.006) and a positive impact on a holistic scale of teacher pedagogical content knowledge (effect size = 0.41, p = 0.026). The study also yielded positive and significant impacts on the amount of time teachers spent on science instruction (effect size = 0.40, p = 0.015) and on the emphasis that teachers placed on NGSS-aligned instructional practices, with statistically significant effect sizes ranging from 0.40 to 0.49. This suggestive evidence that Making Sense of SCIENCE changes classroom science learning experiences in ways that align with expectations in NGSS, which is a hypothesized precursor to measuring impacts on student achievement, deserves notice. In regard to student science achievement, the study did not find statistically significant results for the full sample of students (effect size = 0.06, p = 0.494), or for the sample of students in the bottom third of incoming math and English Language Arts achievement, with effect sizes of 0.22 (p = 0.099) and 0.073 (p = 0.567), respectively. Notably, with the exception of one negative effect size, additional analyses on student achievement using different measures and samples yielded positive, but not statistically significant, effect sizes ranging from 0.02 to 0.12. Evaluators offer three potential contributors to the findings of limited impact on student science achievement. First, given the timing of the study in relation to the release of the NGSS in 2013, finding a suitable NGSS-aligned student science assessment was a challenge. Second, most study schools and districts had not yet adopted NGSS-aligned curricula and did not have access to NGSS-aligned curriculum resources. Third, the sample of teachers was unstable across the two years, with the percentage of teachers leaving the school congruous to the percentage observed at the national level. [For the appendices, see ED609254; for the research summary, see ED609256.]

Purpose: The Santa Ana Unified School District received an i3 grant (Investing in Innovation; funded by the U.S. Department of Education Grant number U411C160074) to build on their existing school climate practices. Called the Enhanced Positive School Climate Model, the aim is to improve school climate, student-adult relationships, create social emotional learning programs, and provide students the needed structure to access challenging curriculum and expectations. As part of this enhanced model, School Climate Liaisons (SCLs) were hired to support and provide coaching to schools in the intervention group on PBIS [Positive Behavioral Intervention and Supports], RP [Restorative Practice] implementation, and behavioral interventions. As part of the i3 grant requirements, the district employed a rigorous cluster-randomized trial whereby half of the district's elementary schools were assigned to receive the Enhanced Positive School Climate Model and half were assigned to a waitlist control. The confirmatory research question is: Do students in grades 4 and 5 at follow-up, in schools assigned to receive support from PBIS and Restorative Practice school climate specialists and community liaisons, who receive services for two years, exhibit higher levels of self-management at the end of the second year, as compared to students in grades 4 and 5 in schools assigned to not receive such support? Additionally, there were exploratory research questions examining changes in other SEL [social-emotional learning] measures including growth mindset, self-efficacy, and social awareness. Methods: This study employed a cluster-randomized controlled design whereby half of the study schools were assigned to the treatment condition and the other half of schools were assigned to the "business as usual" control condition. In total, 35 schools (30 K-5 schools; 5 K-8 schools) were randomly assigned to either the treatment or control condition. All students within these schools were included in the evaluation. Students in the treatment schools (n = 17) were eligible to receive services through the grant. Randomization occurred in May 2017. The 2016-17 school year served as a baseline school year (i.e., no services were provided). The Enhanced Positive School Climate Model began in the 2017-18 school year and continued into the 2018-19 school year for treatment schools only. Control schools were waitlisted and began receiving services in the 2019-20 school year. Students enrolled in district schools in May 2017 were tracked using student roster information through the end of the 2018-19 school year. Only students who were in study schools continuously from the 2016-2017 to 2018-2019 school years were included in the impact analysis. The confirmatory and exploratory outcomes referenced below were conducted using extant data from the district. Data were used from routine data collection processes within the district from the annual CORE climate student survey. The primary outcome measure was the five-item self-management scale on the CORE District's student survey (Taylor, Oberle, Durlak, & Weissberg, 2017; Transforming Education, 2014; https://www.rand.org/education-and-labor/projects/assessments/tool/2014/panorama-social-emotional-learning-questionnaire-measures.html) asked of all students in grades 4-12 each year. For the confirmatory research question (and all exploratory questions), adjusted post-intervention outcomes for students in treatment schools were compared to the outcomes for their counterparts in the control schools. This involved fitting conditional multilevel regression models (i.e., hierarchical linear modeling [HLM]), with additional terms to account for the nesting of individuals within schools (see Goldstein 1987; Raudenbush & Bryk, 2002; Murray 1998). Results: Eight two-level models were estimated per grade level to address the confirmatory and exploratory research questions. Although the students in treatment schools had slightly higher self-management, growth mindset and self-efficacy scores at follow-up, only the difference in fourth grade self-efficacy scores was statistically significant. In contrast, students in control schools had very slightly higher social awareness scores; this finding was not statistically significant. Implications: Meaningful and statistically significant differences were not found between students who attended schools that received the Enhanced Positive School Climate Model compared to their peers in schools that did not receive the Enhanced Positive School Climate Model. The lack of findings may be due to the other changes in school climate practices throughout the district during the study period. Additionally, variations in the implementation of the Enhanced Positive School Climate Model may have contributed to the lack of findings. Although the findings are not significant, the direction of results is mostly consistent, indicating increases in SEL competencies for students receiving additional school climate supports. Future studies should continue to investigate the relationship between whole-school approaches to school climate and changes in SEL competencies for students.

Dual-credit courses expose high school students to college-level content and provide the opportunity to earn college credits, in part to smooth the transition to college. With the Tennessee Department of Education, we conduct the first randomized controlled trial of the effects of dual-credit math coursework on a range of high school and college outcomes. We find that the dual-credit advanced algebra course alters students' subsequent high school math course-taking, reducing enrollment in remedial math and boosting enrollment in precalculus and Advanced Placement math courses. We fail to detect an effect of the dual-credit math course on overall rates of college enrollment. However, the course induces some students to choose four-year universities instead of two-year colleges, particularly for those in the middle of the math achievement distribution and those first exposed to the opportunity to take the course in eleventh rather than twelfth grade. We see limited evidence of improvements in early math performance during college.

As the New York City economy becomes increasingly reliant on workers who have some postsecondary education or training in a specialized field, there is a growing need for local policymakers and educators to identify the most efficient ways to prepare high school students to take on these "middle-skill jobs." These needs are particularly acute in the transportation and manufacturing industries. To address these needs, the New York City Department of Small Business Services (SBS) and Department of Education (DOE) created Scholars at Work (SAW), a program available to an eligible subset of New York City high school students enrolled in Career and Technical Education (CTE) programs. The goal of the SAW program is to expose students to career opportunities, to provide them with real-life work experience alongside adults, and to develop their workplace skills. This report presents findings from the RAND Corporation's evaluation of the SAW program. The evaluation has two components: an implementation study that examines and describes SAW's activities and processes, to understand the extent to which those are functioning as the designers and implementers of the program expect, and an outcomes study, which analyzes how SAW participants are faring in the labor market compared to comparable NYC public school graduates. [This report was supported by the New York City's Mayor's Office for Economic Opportunity.]

As the New York City economy becomes increasingly reliant on workers who have some postsecondary education or training in a specialized field, there is a growing need for local policymakers and educators to identify the most efficient ways to prepare high school students to take on these "middle-skill jobs." These needs are particularly acute in the transportation and manufacturing industries. To address these needs, the New York City Department of Small Business Services (SBS) and Department of Education (DOE) created Scholars at Work (SAW), a program available to an eligible subset of New York City high school students enrolled in Career and Technical Education (CTE) programs. The goal of the SAW program is to expose students to career opportunities, to provide them with real-life work experience alongside adults, and to develop their workplace skills. This report presents findings from the RAND Corporation's evaluation of the SAW program. The evaluation has two components: an implementation study that examines and describes SAW's activities and processes, to understand the extent to which those are functioning as the designers and implementers of the program expect, and an outcomes study, which analyzes how SAW participants are faring in the labor market compared to comparable NYC public school graduates. [This report was supported by the New York City's Mayor's Office for Economic Opportunity.]

Pre-college advising programs exist in most disadvantaged high schools throughout the United States. These programs supplement traditional advising by high school guidance counselors and attempt to help underrepresented and disadvantaged students overcome the complexities of the postsecondary admission and financial aid processes. Existing evidence on these programs often uses within-school randomization where spillovers and alternative supports may confound estimates. We provide the first evidence on a whole school intervention resulting from a school-level randomized controlled trial in the United States. The college access program we study uses a near-peer model where a recent college graduate works at the school assisting students in the application and enrollment process. Pooled results across the first three years of program implementation find no significant impacts on overall college enrollment. However, subgroup analyses reveal positive, significant effects among the groups most targeted by the intervention: Hispanic and low-income students. Most of the impact comes through increasing two-year college enrollment, but this appears to be new entrants rather than inducing students to move from four-year to two-year colleges. The observed positive effects for these subgroups attenuate over time. We attribute this drop in the estimated impact to departures in fidelity of the experiment. Even among the cohorts for which we find positive enrollment impacts, we find no significant impacts on college persistence. [For the grantee submission, see ED594418.]

Nearly two dozen states now administer online exams to deliver testing to K-12 students. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students’ learning. We examine the rollout of computerbased testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the state administering the same exam (PARCC) in online and offline formats each year during the transitional period. We find an online test penalty of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA), which partially but not fully fades out in the second year of online testing.

Building on previous research showing how well ninth-grade student behaviors predict on-time high school graduation, this experimental study investigates the impact of a ninth-grade intervention on student attendance and course passing. The study, conducted in 41 geographically and demographically diverse high schools within a single state, evaluates the effects of placing a half-time staff member in high schools to implement the Early Warning Intervention (EWI) Team model designed to monitor ninth-grade early warning indicators and provide timely interventions. Analyses based on the prespecified student outcomes of attendance rate and percentage of ninth-grade course credits earned indicated no statistically significant impact of the intervention. On secondary outcome variables, results indicated that students in treatment schools were significantly less likely than control school students to be chronically absent. The difference between treatment and control school students on dichotomous measures of course failure were not statistically significant. The widespread dissemination of research and best practices related to early warning systems and ninth-grade interventions likely accounted for low levels of contrast between treatment and control school practices and outcomes.

The three-tiered Positive Behavioral Interventions and Supports (PBIS) framework promotes the development of systems and data analysis to guide the selection and implementation of evidence based practices across multiple tiers. The current study examined the effects of universal (tier 1) or school-wide PBIS (SW-PBIS) in one state's scale-up of this tier of the framework. Annual propensity score weights were generated to examine the longitudinal effects of SW-PBIS from 2006–07 through 2011–12. School-level archival and administrative data outcomes were examined using panel models with an autoregressive structure. The sample included 1316 elementary, middle, and high schools. Elementary schools trained in SW-PBIS demonstrated statistically significantly lower suspensions during the fourth and fifth study years (i.e., small effect size) and higher reading and math proficiency rates during the first two study years as well as in one and two later years (i.e., small to large effect sizes), respectively. Secondary schools implementing SW-PBIS had statistically significantly lower suspensions and truancy rates during the second study year and higher reading and math proficiency rates during the second and third study years. These findings demonstrate medium effect sizes for all outcomes except suspensions. Given the widespread use of SW-PBIS across nearly 26,000 schools in the U.S., this study has important implications for educational practices and policies.

Integrated Planning and Advising for Student Success (iPASS) is an initiative funded by the Bill & Melinda Gates Foundation to support colleges seeking to incorporate technology into their advising and student services. In iPASS, such technology is intended to increase advising's emphasis on a student's entire college experience, enabling advisors to more easily: (1) intervene when students show early warning signs of academic and nonacademic challenges; (2) regularly follow up as students progress through college; (3) refer students to tutoring and other support services when needed; and (4) provide personalized guidance that reflects students' unique needs. MDRC and CCRC partnered with three institutions already implementing iPASS--California State University, Fresno; Montgomery County Community College; and the University of North Carolina at Charlotte--to study how technology can support advising redesign, employing a randomized controlled trial research design. The three institutions increased their emphasis on providing timely support, boosted their use of advising technologies, and used administrative and communication strategies to increase student contact with advisors. The enhancements generally produced only a modestly different experience for students in the program group compared with students in the control group, although at one college, the enhancements did substantially increase the number of students who had contact with an advisor. Consequently, it is not surprising that the enhancements have so far had no discernible positive effects on students' academic performance. Nonetheless, some staff members indicated in interviews that implementing and enhancing iPASS has enabled their institution to take steps toward a stronger system to support students and help them succeed.

The Parent Leadership Institute (PLI) of Children's Aid (CA), funded via a 2013 Investing in Innovation (i3) development grant operated between the 2014-15 through 2018-19 school years. Key goals of the PLI included: (1) improving the capacity of parents to effectively engage in the school community in support of their child; and (2) increasing the capacity of school staff to create and support environments which are welcoming to and supportive of the active engagement of parents as key members of the school community. Through implementation of the PLI, CA expanded its partnership with six schools located in the South Bronx community of Morrisania, an area characterized by high levels of poverty, health disparities, and crime, and low levels of academic achievement and attainment among both children and adults. This report serves as the final report on this phase of the PLI and includes an exploration of implementation during year 4 (2016-17 school year) and analyses of quantitative data on student academic performance. [This report was prepared for Children's Aid New York.]

The current paper reports long-term treatment impact estimates for a randomized evaluation of an early childhood intervention designed to promote children's developmental outcomes and improve the quality of Head Start centers serving high-violence and high-crime areas in inner-city Chicago. Initial evaluations of end-of-preschool data reported that the program led to reductions in child behavioral problems and gains in measures of executive function and academic achievement. For this report, we analyzed adolescent follow-up data taken 10 to 11 years after program completion. We found evidence that the program had positive long-term effects on students' executive function and grades, though effects were somewhat imprecise and dependent on the inclusion of baseline covariates. Results also indicated that treated children had heightened sensitivity to emotional stimuli, and we found no evidence of long-run effects on measures of behavioral problems. These findings raise the possibility that developing programs that improve on the Head Start model could carry long-run benefits for affected children. [This paper was published in "PLOS ONE" Jul 2018.]

"INSPIRE" is an Investing in Innovation (i3) development grant funded by the Office of Innovation and Improvement, U.S. Department of Education. "INSPIRE" provides an innovative integrated K-12 STEM pipeline approach focused on STEM course content and instructional redesign. The INSPIRE model was implemented in Cabarrus County Schools (CCS), which is among the largest school systems in North Carolina, serving nearly 30,000 students in 39 schools. The impact evaluation included two studies that examined the effect of INSPIRE on mathematics and science achievement as measured by North Carolina standardized End-of-Grades assessments. The elementary study (Study 1) used a three-year, longitudinal, single-cohort quasi-experimental design (QED) to assess the impact of INSPIRE on math achievement at the end of 5th grade after two years of program exposure. The secondary study (Study 2) used an individual-level, longitudinal, randomized controlled trial (RCT) with blocking by school level and cohort to assess the effects of INSPIRE on math and science achievement at the end of 7th and 10th grades after two years of program exposure. For both studies, we compared the outcomes of INSPIRE students with similar students from schools that did not offer a STEM program. For the elementary study, propensity score matching (PSM) was used to match INSPIRE elementary students and comparison student samples at baseline (on pre-test math achievement scores, gender, minority status, and economically disadvantaged status) and baseline equivalence was established all pre-test assessment measures; this study met What Works Clearinghouse (WWC) Group Design Standards with Reservations. For the secondary study, the overall and differential attrition rates were low based on the WWC attrition standards (WWC, 2017); this study met WWC Group Design Standards without Reservations. The results of the elementary study indicated a statistically significant difference between the "INSPIRE" treatment group and the business-as-usual comparison group on the math achievement outcome. Comparison students reported a statistically significant higher increase in math achievement than "INSPIRE" students. Results of the secondary study indicated no statistically significant difference between the "INSPIRE" treatment group and the business-as-usual comparison group on the math and science achievement outcome. The duration of students' exposure to INSPIRE, fidelity of implementation, alignment between PBL instruction and NC standardized assessments, and contextual factors that might have weakened the intervention strength relative to business-as-usual conditions are discussed as possible factors that account for these findings. The report concludes with suggestions for future research and implications for education policy.

The Middle-Grades Leadership Development (MLD) Project was designed to develop principal leaders and leadership teams who create high-performing middle-grades schools. Designed by the National Forum to Accelerate Middle-Grades Reform, the four-year project was funded from 2013 to 2017 by a U.S. Department of Education Investing in Innovation (i3) development grant. The project was implemented in 12 middle-grades schools in rural and small town areas of Kentucky and Michigan. Schools received an extensive set of school improvement supports, including: creating a vision using the Forum's Schools to Watch (STW) criteria; engaging in an assessment and planning process for improvement; STW leadership coach; principal mentor; STW mentor schools; leadership team; networking opportunities; and focused professional development. The evaluation of the MLD Project used a quasi-experimental design (QED) with matched comparison schools (12 treatment schools and 38 comparison schools) to examine the impact of the project on intermediate outcomes such as culture, collaboration, work climate, and teaching efficacy, as well as the long term outcomes of principal effectiveness and student achievement. Results showed that MLD treatment schools significantly improved their collaboration practices, teaching efficacy, middle-grades instructional practices, and their implementation of the STW criteria for high performance. There was significant improvement in the long-term outcome of principal effectiveness among treatment principals, with nine of the twelve principals improving their leadership skills and behaviors to the proficient or distinguished levels by the end of the grant. Although there was no overall intervention effect on ELA/reading or math student achievement, seven treatment schools displayed larger growth than the state average for some groups of students. The results provide unique insight into a middle-grades program focused on principal leaders and collaborative leadership. A roadmap that depicts the key supports, activities, and practices implemented at MLD schools that were the most impactful on building middle-grades leadership effectiveness were articulated. These include school-level practices (i.e., guiding vision, continuous improvement practices, reflective practices), principal-level practices (i.e., knowledge of young adolescents, commitment to developmentally appropriate practices, instructional leadership), collaborative leadership practices (i.e., developing teacher leaders, shared capacity), and teaching practices (i.e., student centered, high expectations, rigorous instruction) that combined, result in middle-grades leadership that is more effective. [The report was prepared by the Center for Prevention Research and Development.]

As originally conceptualized, Early Colleges were small schools focused purposefully on college readiness for all students. Frequently located on college campuses, Early Colleges targeted students who might face challenges in postsecondary education, including students who were the first in their family to go to college, economically disadvantaged students, English Language Learners (ELL), or students who are members of racial or ethnic groups underrepresented in college. The Early College Expansion Partnership (ECEP) is among the first large-scale effort to apply Early College strategies into comprehensive high schools. Supported by a $15 million grant from U.S. Department of Education's Investing in Innovation (i3) program, the ECEP was designed to increase the number of students graduating from high school prepared for enrollment and success in postsecondary education. The project sought to blend high school and college by applying strategies from the successful Early College high school model to 14 middle schools, 12 high schools, and two 6th-12th-grade schools in three districts in two states: Colorado and Texas. ECEP implemented an adapted version of the Early College High School Model. The program provided a set of services that supported implementation of a whole-school reform model emphasizing the creation of a college-preparatory school environment. A primary emphasis of the program was increasing the number of students who participated in college-credit-bearing courses while in high school. This report describes the approach used to assess student impacts and to track changes over time; uses survey and site visit data to describe key changes that have been made at the district and school levels; presents the impact estimates for the core student-level outcomes; places the findings in context and discusses the broader implications of this work; and summarizes the overall findings. [For the companion report, "Implementation Supports of the Early College Expansion Partnership," see ED618696.]

The National Math + Science Initiative's (NMSI's) College Readiness Program (CRP) is an established program whose goal is to promote science, technology, engineering, and mathematics education in high schools to improve students' readiness for college. The program provides teacher, student, and school supports to promote high school students' success in mathematics, science, and English Advanced Placement (AP) courses, with a focus on students who are traditionally underrepresented in the targeted AP courses. Through a scale-up grant awarded to NMSI by the Investing in Innovation (i3) program, the CRP was implemented in 28 schools in the 2016-2017 school year. CRESST conducted an independent evaluation of the impact of the CRP on students' AP outcomes using a randomized cluster trial with 28 CRP schools and 24 control schools in 10 states. The evaluation of the CRP consisted of two parts: (1) assessment of the program's impact on selected student AP exam outcomes and (2) assessment of the fidelity of implementation of the CRP. Program impact was evaluated using a 2-level hierarchical generalized linear model (HGLM) with students nested within schools The descriptive statistics showed that a higher percetange of students in the treatment schools took at least one AP course (30.7%) compared to those in the control schools (26.4%) by approximately 4.3%, however the difference was not statistically significant. In addition, students in the treatment schools were not more likely to achieve a score of 3 or higher, when compared to the delayed treatment schools. We further examined the effectiveness of the CRP using the prior year's school-level performance on the AP exam as a covariate. As with the above findings, the results indicated the probability of a student taking at least one AP course or scoring 3 or higher on at least one AP exam is not statistically different between students in the treatment schools and those in the control treatment schools. Fidelity of implementation was evaluated using a fidelity matrix approach (required as part of the evaluation of the i3 program), which showed that not all elements of the program were implemented with high fidelity. Overall results, however, indicated that 23 schools out of 28 treatment schools (82.1%) achieved 80% or better implementation fidelity, for an average fidelity score of 89.5%. Seven schools achieved a perfect 100% fidelity score. Looking at the different indicator groups (school, teacher and student), we found that all school support measures across all schools were implemented with fidelity. In over 80% of schools, not all teachers fulfilled their requirements for attending all training sessions, and so this component was not implemented with fidelity. Stipends and teacher awards were paid as expected as were student award payments. Teacher survey data indicated that teachers found the training and professional development activities provided by the CRP to be the most beneficial program supports relating to helping increase student achievement in AP courses. Teacher incentives were chosen as the least important program component relating to increasing student performance by 16% of teachers and student incentives by 12% of teachers. Teachers did, however, view the student incentives as an important program component to encourage enrollment in AP courses. Likewise, students rated the financial incentives on average as somewhat important in encouraging them to participate in AP courses.

Although high school graduation rates are rising--the national rate was 82 percent during the 2013/14 school year (U.S. Department of Education, 2015)--dropping out remains a persistent problem in the Midwest and nationally. Many schools now use early warning systems to identify students who are at risk of not graduating, with the goal of intervening early to help students get back on track for on-time graduation. Although research has guided decisions about the types of data and indicators used to flag students as being at risk, little is known about the impact of early warning systems on students and schools--and in particular, whether these systems do help get students back on track. This study, designed in collaboration with the REL Midwest Dropout Prevention Research Alliance, examined the impact and implementation of one early warning system--the Early Warning Intervention and Monitoring System (EWIMS)--on student and school outcomes. To assess the impact of EWIMS on student and school outcomes, 73 high schools in three Midwest Region states were randomly assigned to implement EWIMS during the 2014/15 school year (37 EWIMS schools) or to continue their usual practices for identifying and supporting students at risk of not graduating on time and to delay implementation of EWIMS until the following school year (36 control schools). The study included 37,671 students in their first or second year of high school, with 18,634 students in EWIMS schools and 19,037 students in control schools. EWIMS and control schools and students were similar on all background characteristics prior to random assignment. The study examined the impacts of EWIMS on indicators of student risk and on student progress in school after the first year of EWIMS adoption. The study found that EWIMS reduced the percentage of students with risk indicators related to chronic absence and course failure but not related to low GPAs or suspension: (1) The percentage of students who were chronically absent (missed 10 percent or more of instructional time) was lower in EWIMS schools (10 percent) than in control schools (14 percent); this 4 percentage point difference was statistically significant; and (2) The percentage of students who failed one or more courses was lower in EWIMS schools (21 percent) than in control schools (26 percent); this 5 percentage point difference was statistically significant; (3) The percentage of students who had a low GPA (2.0 or lower) was 17 percent in EWIMS schools and 19 percent in control schools; this difference was not statistically significant. However, sensitivity analyses that used continuous GPA data instead of the binary risk indicator showed that, on average, GPAs were higher in EWIMS schools (2.98) than in control schools (2.87); this difference was statistically significant; and (4) The percentage of students who were suspended once or more was 9 percent in both EWIMS and control schools; there was no statistically significant difference. EWIMS did not have an impact on student progress in school. That is, there was not a statistically significant difference between EWIMS and control schools in the percentage of students who earned insufficient credits to be on track to graduate within four years (14 percent in both). At the school level, EWIMS did not have a detectable impact on school data culture, that is, the ways in which schools use data to make decisions and identify students in need of additional support. In nearly all participating schools, overall implementation of the EWIMS seven-step process was low, and implementation was challenging. Nevertheless, EWIMS schools were more likely than control schools to report using an early warning system and having a dedicated team to identify and support at-risk students, but EWIMS schools did not differ from control schools in the frequency of data review or the number and type of interventions offered. This report provides rigorous initial evidence that even with limited implementation during the first year of adoption, using a comprehensive early warning system can reduce the percentage of students who are chronically absent or who fail one or more courses. These short-term results are promising because chronic absence and course failure in grades 9 and 10 are two key indicators that students are off track for on-time graduation. However, because the past research linking indicators to on-time graduation is correlational, it is not yet known if improving these indicators leads to improving on-time graduation rates. Also, EWIMS did not have a detectable impact on other measured indicators that are related to students' likelihood of on-time graduation, including low GPAs, suspensions, and earning insufficient credits. Future research is needed to better understand the mechanisms through which EWIMS had an impact on chronic absence and course failure and why EWIMS did not affect other outcomes. In particular, studies could focus on identifying which staff actions and student experiences lead to improved student outcomes. Studies should also examine whether schools achieve improved overall implementation in subsequent years and whether (and how) the observed impacts fade, grow larger, or extend to other risk indicators (low GPAs and suspensions); to intermediate outcomes (including student persistence and progress in school); and to long-term outcomes (including dropout and on-time graduation rates). The following are appended: (1) Planned implementation of the Early Warning Intervention and Monitoring System; (2) Recruitment, random assignment, and study sample; (3) Data collection and analytic methods; (4) Detailed findings and supplementary analyses; and (5) Disclosure of potential conflicts of interest.

Ninth Grade Academies (NGAs)--also called Freshman Academies--have attracted national attention as a particularly intensive and promising approach for supporting a successful transition for high school freshmen. An NGA is a self-contained learning community for ninth-graders that operates as a school within a school. NGAs have four core structural components: (1) a designated separate space within the high school, (2) a ninth-grade administrator who oversees the academy, (3) a faculty assigned to teach only ninth-grade students, and (4) teachers organized into interdisciplinary teams that have both students and a planning period in common. The theory of action behind NGAs is that when these components are employed together, they interact to create a more personalized learning environment where ninth-grade students feel less anonymous and more individually supported. This, in turn, should help students succeed in school and stay on track to high school graduation. NGAs have shown promising results when employed as part of a whole-school reform model, but in these cases schools have received external support from a developer to create and sustain them. A growing number of schools and districts have been experimenting with NGAs on their own, but the little research that exists on their effectiveness is limited to anecdotal accounts. This study, which is based on a quasi-experimental research design, examines the effect of NGAs on students' progress toward graduation, their academic achievement, and their behavior in several school districts in Florida. The sample for this study includes 27 high schools that created NGAs between 2001-2002 and 2006-2007, along with 16 comparison high schools that serve ninth-grade students with similar characteristics as students in the NGA schools. As context for understanding the impact findings, this study also looks at the extent to which the key features of the NGA model were implemented in the NGA schools in the study and how this differs from the structures and supports in the comparison schools. The key finding is that the NGAs in this study do not appear to have improved students' academic or behavioral outcomes (credit earning, state test scores, course marks, attendance, suspensions, or expulsions). The findings also suggest that it can be difficult for schools to fully implement the components of the NGA model without expert assistance: Three years after their creation, only half the NGAs in the study had all four structural components of the model in place. Nationally, school districts continue to create NGAs, and recent efforts to implement them have incorporated various enhancements that are intended to strengthen and improve their implementation, but little is known about their effectiveness. Because students' experience in ninth grade is an important predictor of their future success, these efforts to create and improve NGAs should be examined in future studies. Appended are: (1) Technical Information; and (2) Beyond the Sunshine State: Ninth Grade Academies in Other School Districts. ["Helping Students Make the Transition into High School: The Effect of Ninth Grade Academies on Students' Academic and Behavioral Outcomes" was written with Janet Quint.]

(Purpose) The purpose of this study was to understand if inclusion of computer activities and videos for pre-K and kinder students, with opportunities to view these videos again at home, providing a repeated learning experience with language concepts and vocabulary, improved young students' early reading skill. (Methods) The study followed one cohort of students over two years. Students who were in preschool in the 2014-15 school year and who progressed into kindergarten in the 2015-16 school year were included. The outcome measures for the impact study were tests that gauge students' language development and early reading skills. These included the Woodcock-Johnson III Tests of Achievement subtests: Letter-Word ID and Word Attack, the Peabody Picture Vocabulary Test, and the Test of Language Development-4, Sentence Imitation subtest. (Results) There were no statistically significant differences between ATC and control students in early reading skills, as measured by the confirmatory outcome measure. We also found no statistically significant differences between ATC and control students on additional post-tests analyzed for exploratory purposes. (Implications) The findings of the impact study suggest that the evaluation study may have been underpowered and that the effect of ATC on some measures of early reading skills compared with the business-as-usual condition may be directionally positive on average but less than 0.20 standard deviations. More research is needed with larger sample sizes and lower minimally detectable effect sizes. Another limitation of this study is that the business-as-usual condition consisted of programs that are, at least in some ways, similar to the ATC intervention. Thus, the effect size of ATC may be much greater in a context where ATC participation is compared with a lower quality preschool or kindergarten program. Supplemental tables are appended.

Background: School-based social-emotional and character development (SECD) programs can influence not only SECD but also academic-related outcomes. This study evaluated the impact of one SECD program, Positive Action (PA), on educational outcomes among low-income, urban youth. Methods: The longitudinal study used a matched-pair, cluster-randomized controlled design. Student-reported disaffection with learning and academic grades, and teacher ratings of academic ability and motivation were assessed for a cohort followed from grades 3 to 8. Aggregate school records were used to assess standardized test performance (for entire school, cohort, and demographic subgroups) and absenteeism (entire school). Multilevel growth-curve analyses tested program effects. Results: PA significantly improved growth in academic motivation and mitigated disaffection with learning. There was a positive impact of PA on absenteeism and marginally significant impact on math performance of all students. There were favorable program effects on reading for African American boys and cohort students transitioning between grades 7 and 8, and on math for girls and low-income students. Conclusions: A school-based SECD program was found to influence academic outcomes among students living in low-income, urban communities. Future research should examine mechanisms by which changes in SECD influence changes in academic outcomes.

In 2004, the U.S. Department of Education's Institute of Education Sciences contracted with Mathematica Policy Research to conduct a large-scale evaluation of comprehensive teacher induction. The purpose of the study was to determine whether augmenting the set of services districts usually provide to support beginning teachers with a more comprehensive program improves teacher and student outcomes. This is the study's third and final report on the program's impacts. This report compares retention, achievement, and classroom practices of teachers who were offered comprehensive induction services to teachers who were offered the support normally offered by the school. Teachers assigned to receive comprehensive induction for either one or two years were supported by a full-time mentor who received ongoing training and materials to support the teachers' development. The teachers also were offered monthly professional development sessions and opportunities to observe veteran teachers. The teachers were followed for three years. Data was collected from 1,009 beginning teachers in 418 schools in 17 districts. Districts included in the study were not already offering comprehensive induction services, including paying for full-time mentors. Novice teachers in approximately half of the schools were assigned by lottery to receive comprehensive induction services. In 10 of the districts, these teachers were provided one year of comprehensive induction services; in the remaining 7 districts, the teachers were provided two years of services. Teachers in the schools not assigned to receive comprehensive induction services were provided the support normally offered to novice teachers by the school. Teacher practices were measured via classroom observations conducted in the spring of 2006. Data on teacher retention were collected via surveys administered in the fall of 2006, 2007, and 2008. Student test scores were collected from district administrative records for the 2005-06, 2006-07, and 2007-08 school years. Key findings include: (1) During the comprehensive induction program, treatment teachers received more support than control teachers; (2) The extra induction support for treatment teachers did not translate into impacts on classroom practices in the first year; (3) For teachers who received one year of comprehensive induction, there was no impact on student achievement; (4) For teachers who received two years of comprehensive induction, there was no impact on student achievement in the first two years. In the third year, there was a positive and statistically significant impact on student achievement; and (5) Neither exposure to one year nor exposure to two years of comprehensive induction had a positive impact on retention or other teacher workforce outcomes. The following are appended: (1) Supplemental Information for Chapters II and III; (2) Supplemental Information for Chapter IV; (3) Sensitivity Analyses and Supplemental Information for Chapter V; and (4) Sensitivity Analyses and Supplemental Information for Chapter VI.

In low-performing public high schools in U.S. cities, high proportions of students drop out, students who stay in school typically do not succeed academically, and efforts to make substantial reforms often meet with little success. The Talent Development High School model is a comprehensive school reform initiative that has been developed to address these challenges. Targeting some of the most troubled schools in the country, the model seeks to raise the expectations of teachers and students and to prepare all students for postsecondary education and employment. MDRC, a nonpartisan, nonprofit education and social policy research organization, conducted an independent, third-party evaluation of Talent Development. This rigorous evaluation focuses on the first five high schools to begin using the model in the School District of Philadelphia. The evaluation follows 20 cohorts of ninth-grade students for up to four years of high school using a comparative interrupted time series research design. Appended are: (1) Tables for First-Time Ninth-Grade Students; and (2) Tables for Repeating Ninth-Grade Students. (Contains 16 tables, 9 figures, and 3 boxes.)[Dissemination of MDRC publications is also supported by Starr Foundation.]

Policymakers have long been concerned about the disparities in college attendance between more and less advantaged groups of students. Upward Bound is one of the largest and longest running federal programs designed to help economically disadvantaged students prepare for, enter and succeed in college. Since December 1991, Mathematica Policy Research, Inc., (MPR) has been conducting the national evaluation of Upward Bound for the U.S. Department of Education (ED). The evaluation has focused on program implementation issues and the effects of the program on student outcomes. The "impact study" is designed to measure the impacts or effects of regular Upward Bound on student outcomes, and it is based on a longitudinal evaluation in which eligible applicants from a nationally representative sample of projects were randomly assigned to Upward Bound or to a control group. The results in this document are based on the national evaluation's third follow-up data collection, which was completed in 2000. Because the entire sample of students was beyond high school age by that time, the report includes updated findings on the effects of Upward Bound on high school outcomes. In addition, based on data covering the first few years after sample members left high school, the report addresses the following research questions: (1) What effect does Upward Bound have on students' postsecondary experiences? (2) Who benefits most from Upward Bound? and (3) What is the association between staying in Upward Bound and student outcomes? Findings in this report suggest that for the average student, Upward Bound: (1) increased the number of high school math credits earned by participants; (2) did not affect other measures of high school academic preparation; (3) may have increased enrollment at four-year institutions; and (4) did not affect enrollment at postsecondary institutions more generally when all types of postsecondary institutions are considered. Appended are: (1) Sample Design Unit Nonresponse and Weights; (2) Baseline Characteristics of the Treatment and Control Groups, Third Follow-Up Survey Respondents; (3) Program Effects and Standard Errors; (4) The Effect of Upward Bound on High School Outcomes by Selected Subgroups; (5) The Effect of Upward Bound on College Engagement by Selected Subgroups; (6) Methods Used to Estimate the Effects of Additional Upward Bound Participation; (7) Weighted Standard Deviations for All Outcome Variables; (8) Data Collection; and (9) Sample Sizes and Standard Errors for Reported Impact Estimates. (Contains 29 tables and 3 figures.)

This study sought to evaluate the effectiveness of a two-quarter, extended orientation program for new, undecided students at the University of California, Irvine (UCI). The course was designed to assist students with the transition from high school to college and acquaint them with strategies to maximize their success at UCI. Of the 690 new, unaffiliated freshmen who entered UCI in fall quarter 1993, 24 enrolled in the extended orientation course for both quarters, 34 enrolled in fall only, 30 enrolled in the winter quarter, and 602 enrolled in neither quarter. The study found that, compared with other unaffiliated students who did not enroll in the extended orientation courses, those who enrolled for one or more quarters obtained significantly higher grade point averages (GPAs) and units completed by the end of spring quarter. Approximately 90 percent of students surveyed who expressed satisfaction with the course said that they would recommend it to other freshmen. (MDM)

The cooperative elementary school model uses cooperation, particularly cooperative learning, as a philosophy for educational change. A 2-year study of the cooperative elementary school model in 2 treatment and 3 comparison schools involving 1,012 students demonstrates positive effects on academic achievement and social relations. (SLD)