Search Results: (1-15 of 60 records)
|REL 2017272||Getting students on track for graduation: Impacts of the Early Warning Intervention and Monitoring System after one year
Early warning systems that use research-based warning signs to identify students at risk of dropping out of high school have emerged as one strategy for improving graduation rates. This study tested the impact of one early warning system, the Early Warning Intervention and Monitoring System (EWIMS), on 37,671 students in grades 9 and 10 and their schools after one year of implementation. Seventy-three high schools were randomly assigned to implement EWIMS during the 2014/15 school year or to continue their usual practices for identifying and supporting students at risk of not graduating on time. Impact findings show that EWIMS reduced the percentage of students with risk indicators related to chronic absence and course failure but not related to low grade point averages, suspensions, or insufficient credits to graduate. At the school level, EWIMS did not have a detectable impact on school data culture, that is, the ways in which schools use data to make decisions and identify students in need of additional support. Findings suggest that overall implementation of the EWIMS seven-step process was low in nearly all EWIMS schools, and that implementation of EWIMS was challenging for participating schools. The authors hypothesize that other school-level processes, unmeasured in this study, also may have contributed to impacts on students. For example, effects might have emerged for chronic absence and course failure if schools prioritized encouraging students to show up and participate in their courses, even if they did not have a sophisticated set of interventions. Further research is needed to better understand the mechanisms through which EWIMS had an impact on chronic absence and course failure. This report provides rigorous, initial evidence that even with limited implementation during the first year of adoption, use of a comprehensive early warning system such as EWIMS can reduce the percentage of students who are chronically absent or who fail one or more courses. These short-term results are promising because chronic absence and course failures in grades 9 and 10 are two key indicators that students are off track for graduation.
|REL 2017168||Stated Briefly: A comparison of two approaches to identify beating-the-odds high schools in Puerto Rico
This "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. The Regional Educational Laboratory Northeast and Islands conducted this study using data on public high schools in Puerto Rico from national and territory databases to compare methods for identifying beating-the-odds schools. Schools were identified by two methods, a status method that ranked high-poverty schools based on their current observed performance and an exceeding-achievement-expectations method that ranked high-poverty schools based on the extent to which their actual performance exceeded (or fell short of) their expected performance. Graduation rates, reading proficiency rates, and mathematics proficiency rates were analyzed to identify schools for each method. The identified schools were then compared by method to determine agreement rates—that is, the amount of overlap in schools identified by each method. The report presents comparisons of the groups of schools—those identified by each method and all public high-poverty high schools in Puerto Rico—on descriptive information. Using the two methods—ranking by status and ranking by exceeding-achievement-expectations—two different lists of beating-the-odds schools were identified. The status method identified 17 schools, and the exceeding-achievement-expectations method identified 15 schools. Six schools were identified by both methods. The agreement rate between the two lists of beating-the-odds schools was 38 percent. The analyses suggest that using both methods to identify beating-the-odds schools is the best strategy because high schools identified by both methods demonstrate high levels of absolute performance and appear to be achieving higher levels of graduation rates and percent proficiency than might be expected given their demographics and prior performance.
|NCEE 20174001||Race to the Top: Implementation and Relationship to Student Outcomes
Race to the Top (RTT), one of the Obama administration's signature programs and one of the largest federal government investments in an education grant program, received $4.35 billion in funding as part of the American Recovery and Reinvestment Act of 2009. Through three rounds of competition in 2010 and 2011, RTT awarded grants to states that agreed to implement a range of education policies and practices designed to improve student outcomes. Using 2013 interview data from all states, this report documents whether states that received an RTT grant used the policies and practices promoted by RTT and how that compares to non-grantee states. The report also examines whether receipt of an RTT grant was related to improvements in student outcomes. Findings show that 2010 RTT grantees reported using more policies and practices than non-grantees in four areas (standards and assessments, teachers and leaders, school turnaround, charter schools), and 2011 RTT grantees reported using more in one area (teachers and leaders). However, the relationship between RTT and student outcomes was not clear, as trends in test scores could be plausibly interpreted as providing evidence of either a positive, negative, or null effect for RTT.
|REL 2016118||Identifying early warning indicators in three Ohio school districts
The purpose of this study was to identify a set of data elements for students in grades 8 and 9 in three Ohio school districts that could serve as accurate early warning indicators of their failure to graduate high school on time and to comparatively examine the accuracy of those indicators. In order to identify the set of indicators with the optimal accuracy for each district, the research team collected student-level data on two cohorts of grade 8 and 9 students in each school district. Datasets used in the analyses included students’ four-year graduation status (the outcome) and 8th and 9th grade data on attendance, coursework, suspensions, and test score records (the candidate early warning indicators). Logistic regression and Receiver Operating Characteristic (ROC) curve analyses were used to identify the candidate indicators that were the consistent predictors of students’ failure to graduate on time in each district and to identify the cut points on the original scales that most accurately distinguish students who were at risk of not graduating on time from those who did graduate on time. The analyses were restricted to students who were first-time freshmen within the districts in 2006/07 or 2007/08, and excluded students who entered the district after grade 9. Students in the 2006/07 cohort graduated in 2010, and students in the 2007/08 cohort graduated in 2011. The three districts included in the study varied in size, demographic composition, and locale. Results show that the optimal cut point for classifying students as at risk varied significantly across districts for five of the eight candidate indicators included in the study. Across the three districts and two grades, different indicators were identified as the most accurate predictors of students’ failure to graduate on time. End-of-year attendance rate was the only indicator that was a consistent predictor for both grades in all three districts. The most accurate indicators in both grade 8 and grade 9 were based on coursework (GPAs and course credits). Consistent with prior literature, failing more than one class and earning one or more suspensions also were strong predictors of failure to graduate on time. On average, indicators were more accurate in grade 9 than in grade 8. Findings illustrate why it is important for districts to conduct local validation using their own data to verify that indicators selected for their early warning systems accurately predict students’ failure to graduate on time. The methods laid out in this study can be used to help districts identify the best off-track indicators, and indicator cut points, for their particular early warning systems.
|REL 2016141||School reading performance and the extended school day policy in Florida
Beginning with the 2012/13 school year, Florida law required that the 100 lowest-performing elementary schools in reading extend the school day. This study examined how the lowest performing schools implemented the extended school day policy and the trends in school reading performance among the lowest performing schools and other elementary schools. The lowest-performing schools were located throughout Florida and on average, were smaller but served higher proportions of minorities and higher proportions of students receiving free or reduced-price lunch compared to other elementary schools. The lowest-performing schools reported increasing the number of minutes of reading instruction provided to students, increasing staff, and providing different instruction in the extra hour than during other reading instructional blocks. An increase in reading performance was observed for the lowest-performing schools the year the extended school day was implemented. However, this increase did not exceed what would have been expected in the absence of the required increase in reading instruction.
|REL 2015077||Comparing Methodologies for Developing an Early Warning System: Classification and Regression Tree Model Versus Logistic Regression
The purpose of this report was to explicate the use of logistic regression and classification and regression tree (CART) analysis in the development of early warning systems. It was motivated by state education leaders' interest in maintaining high classification accuracy while simultaneously improving practitioner understanding of the rules by which students are identified as at-risk or not at-risk readers. Logistic regression and CART were compared using data on a sample of grades 1 and 2 Florida public school students who participated in both interim assessments and an end-of-the year summative assessment during the 2012/13 academic year. Grade-level analyses were conducted and comparisons between methods were based on traditional measures of diagnostic accuracy, including sensitivity (i.e., proportion of true positives), specificity (proportion of true negatives), positive and negative predictive power, and overall correct classification. Results indicate that CART is comparable to logistic regression, with the results of both methods yielding negative predictive power greater than the recommended standard of .90. Details of each method are provided to assist analysts interested in developing early warning systems using one of the methods.
|REL 2015056||A Practitioner's Guide to Implementing Early Warning Systems
To stem the tide of students dropping out, many schools and districts are turning to early warning systems (EWS) that signal whether a student is at risk of not graduating from high school. While some research exists about establishing these systems, there is little information about the actual implementation strategies that are being used across the country. This report summarizes the experiences and recommendations of EWS users throughout the United States.
|REL 2015054||Review of Research on Student Nonenrollment and Chronic Absenteeism: A Report for the Pacific Region
In some areas of the Regional Educational Laboratory (REL) Pacific Region, between one-fourth and a half of secondary school–age students are not enrolled in school. Not being enrolled in school or being chronically absent can have lasting effects on students’ economic and social development. This REL Pacific report summarizes research on nonenrollment and chronic absenteeism from the United States and emergent nations that share characteristics with Pacific island nations. Four types of factors influence student nonenrollment and absenteeism: student-specific, family-specific, school-specific, and community-specific. Many of these potential factors are interconnected, and the effects of these factors may vary by region. Therefore, educators, policymakers, and family and community members in the Pacific Region may need to gather additional data in order to explore these factors in their own communities. Stakeholders can also use this review to begin to identify the root causes for why students are not in school in order to develop and implement targeted strategies to support student enrollment and attendance.
|REL 2014036||Using evidence-based decision trees instead of formulas to identify at-risk readers
The purpose of this study was to examine whether the early identification of students who are at-risk for reading comprehension difficulties is improved using logistic regression or classification and regression tree (CART). This research question was motivated by state education leaders’ interest in maintaining high classification accuracy while simultaneously improving practitioner understanding of the rules by which students are identified as at-risk or not at-risk readers. Logistic regression and CART were compared using data on a sample of grades 1 and 2 Florida public school students who participated in both interim assessments and an end-of-the year summative assessment during the 2012/13 academic year. Grade-level analyses were conducted and comparisons between methods were based on traditional measures of diagnostic accuracy, including sensitivity (i.e., proportion of true positives), specificity (proportion of true negatives), positive and negative predictive power, overall correct classification, and the receiver operating characteristic area under the curve. Results indicate that CART is comparable to logistic regression, with the results of both methods yielding negative predictive power greater than the recommended standard of .90. The comparability of results suggests that CART should be used due to its ease in interpretation by practitioners. In addition, CART holds several technical advantages over logistic regression.
|REL 2014032||Beating the Odds: Finding Schools Exceeding Achievement Expectations with High-Risk Students
State education leaders are often interested in identifying schools that have demonstrated success with improving the literacy of students who are at the highest level of risk for reading difficulties. The identification of these schools that are “beating the odds” is typically accomplished by comparing a school’s observed performance on a particular exam, such as a state achievement exam, with how the school would be expected to perform when taking into account its demographic characteristics including the percentage of students classified as economically disadvantaged, minority, or as an English language learner. This study used longitudinal data from the Florida Department of Education on grade 3 public school students for the academic years 2010/11-2012/13 to determine which schools are exceeding student achievement expectations, and what demographic similarities exist between schools that are exceeding expectations and other schools.
|REL 2014028||Suspension and Expulsion Patterns in Six Oregon School Districts
This Regional Educational Laboratory (REL) Northwest study identifies how frequently students in six selected urban districts received exclusionary discipline during the 2011/12 school year, the most common reasons for such discipline, the percentage of students receiving multiple suspensions, and how many school days students lost to suspensions. The study also examined the application of exclusionary discipline at different grade spans and by student gender, race/ethnicity, and special education status.
Key findings include:
|WWC SSR10070||WWC Review of the Report "Staying on Track: Testing Higher Achievement's Long-Term Impact on Academic Outcomes and High School Choice"
The 2013 study, Staying on Track: Testing Higher Achievement’s Long-Term Impact on Academic Outcomes and High School Choice, examined the effects of Higher Achievement, a multi-year afterschool and summer program for incoming fifth and sixth graders attending schools in at-risk communities. The program's goal is to improve academic achievement and encourage matriculation into an academically competitive high school. The study included 952 fifth and sixth graders in Washington, DC and Alexandria, Virginia. The researchers found that 4 years after randomization, students who were offered participation in Higher Achievement had significantly higher standardized test scores in mathematical problem solving. They were also significantly more likely than comparison students to be admitted to and matriculate at private high schools, and were less likely to apply to, be admitted to, and matriculate at noncompetitive public charter/magnet schools. No statistically significant differences were found for standardized tests of reading comprehension; application to private schools; application to, admittance to, or matriculation at competitive public charter/magnet schools; or matriculation at neighborhood public schools. This study is a well-executed randomized controlled trial that meets WWC evidence standards without reservations.
|WWC SSR10052||WWC Review of the Report "Assessing the Effectiveness of First Step to Success: Are Short-term Results the First Step to Long-term Behavioral Improvements?"
The 2013 study, Assessing the Effectiveness of First Step to Success: Are Short-term Results the First Step to Long-term Behavioral Improvements?, examined the effects of First Step to Success (First Step), a school- and home-based program intended to improve outcomes for students with moderate to severe behavior problems who may be at risk for academic failure. Researchers randomly assigned 48 elementary schools from across five states to either an intervention group that received the First Step program or a comparison group that received regular instruction. Study authors measured the effects of First Step by comparing parent, teacher, and researcher assessments of student behavior for students in the intervention and comparison groups. While 10 outcomes were measured, only three met WWC evidence standards with reservations: academic engaged time (the proportion of time a student is academically involved), problem behavior, and academic competence. Although the schools were randomly assigned to the intervention and comparison groups, the students who were selected to participate in the study may have differed systematically across the schools. In particular, teachers' selection of the students for the study and the parental consent process both occurred after randomization and, therefore, both of these processes could have been affected by knowledge of the school's research condition. Because of these selection and consent issues, the study was reviewed as a quasi-experimental design by the WWC. The research for the remaining seven outcomes measured did not meet WWC standards.
|WWC IRCWD616||Tools for Getting Along
The Tools for Getting Along program is designed to help elementary school teachers establish a positive, cooperative classroom atmosphere for their students with disabilities. The program is intended to reduce disruptive and aggressive behavior by helping students develop anger management skills. The WWC identified four studies that investigated the effects of Tools for Getting Along on children classified as having an emotional disturbance (or children at risk for classification). None of the studies meet WWC evidence standards. Therefore, the WWC is unable to draw any conclusions based on research about the effectiveness or ineffectiveness of the program. More research is needed to determine if the program works for students with disabilities.
|REL 2012134||Using the Freshman On-Track Indicator to Predict Graduation in Two Urban Districts in the Midwest Region
This report examines how well the freshman on-track indicator developed by the Consortium on Chicago School Research predicts on-time graduation in two urban districts in the Midwest Region. This indicator classifies students at the end of the first year of high school as on-track or off-track to graduate based on grade 9 course credits earned and failures.
REL Midwest examined on-track and off-track rates and for recent freshman cohorts as well as 4-year graduation rates for on-track and off-track students.
Key findings include: