Search Results: (1-14 of 14 records)
|REL 2023143||Encouraging Families to Visit a Literacy Website: A Randomized Study of the Impact of Email and Text Message Communications
The Arkansas Department of Education partnered with the Regional Educational Laboratory Southwest to study the feasibility and effectiveness of using brief email and text message communications to increase the number of parent and guardian visits to the Reading Initiative for Student Excellence (R.I.S.E.) state literacy website.
In November 2021, the department sent test messages to families to determine the percentage of households with children in kindergarten–grade 6 in Arkansas public schools that had a working email address or cell phone number and whether the percentage differed by school locale (rural or nonrural) or demographic composition (percentage of economically disadvantaged students, Black students and Hispanic students, or English learner students). Subsequently, the study team randomly assigned 700 Arkansas public elementary schools to one of eight conditions, which varied the mode of communication (email only or email and text message), the presentation of information (no graphic or with a graphic), and the type of sender (generic sender or known sender). In January 2022 households with children in these schools were sent three rounds of communications with information about literacy and a link to the R.I.S.E. website. The study examined the impact of these communications on whether parents and guardians clicked the link to visit the website (click rate) and conducted an exploratory analysis of differences in how long they spent on the website (time on page).
|REL 2023140||Biliteracy Seals in a Large Urban District in New Mexico: Who Earns Them and How Do They Impact College Outcomes?
New Mexico is one of 48 states that offer a biliteracy seal to high school graduates to recognize their proficiency in a non-English language. The Regional Educational Laboratory Southwest English Learners Research Partnership collaborated with a large urban district in New Mexico to study the characteristics and college readiness of students who earn different types of biliteracy seals (state, district, and global seals) and whether earning a seal improves college outcomes. The study used data from three cohorts of students who graduated from high school in the district from 2017/18 to 2019/20. The study examined the characteristics and college readiness of students who earned different types of seals, the number of students who met some requirements for a seal but did not earn one, and the effect of earning a seal on college outcomes.
|REL 2022138||Effects of Reclassifying English Learner Students on Student Achievement in New Mexico
This study examined how attaining English proficiency and being reclassified as fluent English proficient affected achievement in English language arts and math in the first year after student reclassification in grades 3–8 in New Mexico. State policy in New Mexico bases student reclassification decisions on whether students attain a minimum overall English language proficiency level score of 5.0 on the ACCESS for ELLs (ACCESS) assessment. The study focused on achievement among English learner students in 2014/15–2018/19, a time when the ACCESS underwent a standards setting process to better align its language proficiency scoring scale with the expectations of college- and career-ready standards. After the standards setting, a smaller percentage of English learner students in New Mexico attained English proficiency and were reclassified each year. At the same time, students who scored near the English proficiency level required for reclassification performed above the statewide average in English language arts and math and were more likely to meet state content proficiency standards. However, the study found no effects of reclassification on student achievement either before or after the ACCESS standards setting. In addition, the study found no effect of reclassification on next-year English language arts and math achievement among most groups of students with different characteristics and among most districts in the study. Leaders at the New Mexico Public Education Department could use the findings of this study to consider maintaining the current reclassification threshold. In addition, the state and its districts might want to identify opportunities to strengthen the supports provided to English learner students. This could begin by collecting more systematic information on the education services and supports that English learner students receive leading up to and after they attain English proficiency.
|REL 2022121||Examining the implementation and impact of full-day kindergarten in Oregon
Many states and districts offer full-day kindergarten (FDK) to provide additional time for student learning in the hope that it will improve student outcomes. Prior research has shown an association between FDK and gains in student outcomes such as math and reading standardized assessment scores. In 2015/16, through a policy shift, Oregon changed its funding structure for kindergarten enrollment, which created incentives for districts to offer FDK. This study examines three aspects of FDK in Oregon. First, the study looked at the characteristics of Oregon districts that offered FDK in 2013/14 and 2014/15 (the two years before the policy shift) and how those FDK programs were structured. Next, the study estimated the impact of attending FDK in one large Oregon school district in 2013/14 and 2014/15 on academic and non-academic outcomes. Lastly, the study explored how FDK programs were implemented in 2017/18 (after the policy shift).
Examining how FDK programs were implemented after the policy shift, only 22 percent of teachers responded to a survey, and those respondents reported a focus on teacher-directed activities and limited use of kindergarten entry assessment data in 2017/18. These findings cannot be generalized to all FDK teachers in Oregon and only apply to teachers who responded. In the same year, the 42 percent of principals who responded to the survey reported that a small number of FDK students only received a half day of instruction and that there was a lack of curricular and professional development alignment between preschool and kindergarten. Again, these findings cannot be generalized to all schools with FDK. The study’s mixed findings indicate that FDK may slightly improve student attendance in early elementary grades for some student groups and in settings that are similar to the large district examined in this study. The study also reveals a need for more research on the barriers to offering, accessing, and implementing FDK, as well as the variation in the impact of FDK on student outcomes. Finally, the study points to a need for additional state guidance and support on how to implement high-quality FDK programs.
|REL 2022128||Impacts of Home Visits on Students in District of Columbia Public Schools
This study examined the impacts of structured relationship-building teacher home visits conducted in grades 1–5 as part of a family engagement program in the District of Columbia Public Schools. Using a matched comparison group research design, the study measured the impacts of the home visits on student disciplinary incidents and attendance. The study found that a home visit before the start of the school year reduced the likelihood of a student having a disciplinary incident in that school year. During the school year following a home visit, 9.27 percent of visited students had a disciplinary incident compared with 12.22 percent of nonvisited comparison students. The study also found that, on average, a home visit slightly improved student attendance. The attendance rate averaged 95.28 percent for visited students and 94.93 percent for nonvisited comparison students.
|REL 2022119||What Were the Reach and Impact of the Oregon Promise Financial Aid Program in Its First Two Years?
In 2015 Oregon became the second state in the country to implement a statewide promise program. Its program, Oregon Promise, seeks to promote students' postsecondary attainment by covering nearly all community college tuition. This study used student data from K–12 public schools, Oregon Promise applications, and postsecondary records to examine which public high school seniors the program reached and served and to assess the program's impact on high school graduates' postsecondary outcomes in its first two years. The study found that Oregon Promise applicants generally reflected the demographic composition of all Oregon public high school seniors in 2015/16 and 2016/17, although applicants were more likely to be female and less likely to have received special education services. While applicant characteristics were similar in the first and second years, there were fewer eligible applicants and recipients in the second year, when an expected family contribution limit was added, than in the first year, and they were more likely to be from low-income households and to be students of color. Using grade point average (GPA) data from the Portland metropolitan area, the study also found that lowering the GPA requirement would have increased the overall applicant pool, as well as the number of applicants from low-income households and applicants of color. Just over half of recipients in the first year of the program renewed their award and received it in their second year at a community college. These recipients had better high school attendance and were more likely to have participated in college-level coursework during high school than recipients who received an award only in their first year. Finally, among high school graduates in the Portland metropolitan area with a GPA close to the eligibility cutoff (2.5), the offer of an award had a positive impact on first-year persistence and on persistence or college completion within four years of high school graduation. Findings from the statewide exploratory analysis also found positive program impacts on first-year persistence and persistence or college completion within three or four years of high school graduation for all 2015/16 and 2016/17 seniors in the state. Oregon stakeholders can use the findings to better understand the reach and impact of the Oregon Promise program, implications of program requirements on the number and composition of applicants and recipients, and the high school experiences of recipients who renewed their award.
|REL 2021097||First-Year Effects of Early Indicator and Intervention Systems in Oregon
Although Oregon has made recent gains in its overall high school graduation rate, 21 percent of public school students entering grade 9 in 2014 did not graduate within four years, by 2018. To improve graduation rates, Oregon voters approved Ballot Measure 98 in 2016 to fund dropout prevention and college and career readiness initiatives in high schools. Many districts used the funding to adopt an early indicator and intervention system (EIIS) to identify students who are not on track to graduate on time by monitoring related indicators, such as chronic absenteeism, disciplinary infractions, course progression, and academic performance, through a frequently updated data system. Districts can tailor the system by setting their own on-track thresholds for each indicator to identify students at risk of not graduating on time, assigning those students to interventions, and monitoring student response to the interventions.
This study took advantage of the additional funding being offered to districts across the state to look at first-year effects on chronic absenteeism, disciplinary infractions, course progression, and academic performance by comparing the outcomes in 65 districts that adopted an EIIS to the outcomes in a set of similar districts that used the additional funding for other dropout prevention or college and career readiness initiatives. The study offers insight into the effectiveness of early efforts to scale up EIISs, a popular school-level intervention. EIIS adoption appears to have reduced the percentage of students who were chronically absent by 3.9 percentage points but does not appear to have had positive effects on the three other student outcomes during the first year: the percentage of students with disciplinary infractions, the percentage of grade 10 students who had acquired enough credits by the end of grade 9 to be considered on track for on-time graduation, or the percentage of grade 11 students meeting or exceeding proficiency standards on state math and English language arts tests. The findings offer the Oregon Department of Education information on the early effects of its efforts to promote EIIS across Oregon. The findings can also be used by other state and district education leaders to inform their considerations to scale up EIIS or other similar programs.
|REL 2021111||Professional Development Incentives for Oregon's Early Childhood Education Workforce: A Randomized Study
Many states seek to increase the education levels of their early childhood education (ECE) workforce to improve the quality of care for children. Oregon encourages all ECE workforce members to sign up for a career lattice, a career pathway system that helps them determine goals related to increasing their education. The state also offers incentives for reaching specific steps in the career lattice and scholarships for college credit and community-based training. This study used two randomized controlled trials in 2018 and 2019 to test whether sending emails and offering different financial incentives to Oregon ECE workforce members increased career lattice sign-up and increased education and training levels or workplace retention. The study found that sending emails encouraging career lattice sign-up had no detectable impact on career lattice sign-up or workplace retention. Sending emails offering a monetary incentive at an earlier-than-usual step on the career lattice had a positive impact on training hours recorded but no detectable impact on career lattice movement, college credit hours earned, or workplace retention. Sending emails about automatic enrollment in a scholarship program had no detectable impact on scholarship use, career lattice movement, college credit hours earned, or workplace retention. Lastly, after participants were randomly assigned to study groups, the email campaigns were implemented as planned, reaching all intended participants, although the interventions ended sooner than planned because of a state policy change. The findings suggest that low-touch interventions such as emails have promise for increasing training hours but are not sufficient to induce changes in career lattice sign-up, continuing postsecondary education, or workplace retention for Oregon ECE workforce members. These results have implications for future research, in addition to demonstrating how better messaging and supports can mitigate barriers to further education and training and how email campaigns can be leveraged for workforce communication efforts. This information is particularly relevant for state agencies and education and training providers.
|REL 2021101||The Effect of School Report Card Design on Usability, Understanding, and Satisfaction
Education policymakers view transparency and accountability as critical to the success of schools. To support these goals, the District of Columbia Office of the State Superintendent of Education (OSSE) has developed an online school report card for communicating information about the characteristics and performance of schools. To support OSSE’s interest in making report cards more usable, this study assessed the effect of different designs on how easy the report cards are to use and understand, how easy it is to find information in them, and whether users would recommend the site to others.
The study found that moving the link to details of the district’s School Transparency and Reporting (STAR) framework from the top of the page to beneath the STAR score improved the site’s usability and that reporting the number of points possible for each metric led to a better understanding of how the score is calculated. The combination of design features that produced the best performance on all measures included these two design changes. Other designs had mixed effects. In particular, making year-over-year change in school performance salient made it easier to identify which schools had improved the most, but participants disliked this feature (demonstrated by lower ratings for usability and satisfaction). In general, participants who accessed the site with mobile devices had more difficulty using it. This study illustrates how policymakers and practitioners in other states can efficiently test school report card design changes at scale.
|REL 2021090||Indiana and Minnesota Students Who Focused on Career and Technical Education in High School: Who Are They, and What Are Their College and Employment Outcomes?
In Indiana and Minnesota the state education agency, state higher education agency, and the state workforce agency have collaborated to develop career and technical education courses intended to improve high school students' college and career readiness. These agencies partnered with the Regional Educational Laboratory Midwest to examine whether high school graduates in each state who completed a large number of career and technical education courses in a single career-oriented program of study (concentrators) had different college and workforce outcomes from graduates who completed fewer (samplers) or no career and technical education courses (nonparticipants). The study found that in the 2012/13–2017/18 graduation cohorts, male graduates were more likely to be concentrators than female graduates, and graduates who received special education services were more likely to be concentrators than those who did not receive services. Graduates who were not proficient in reading in grade 8 also were more likely to become concentrators than those who were proficient. Graduates who attended urban and suburban schools were more likely than students who attended town and rural schools to be nonparticipants. Concentrators were less likely than samplers and nonparticipants with similar characteristics to enroll in college, but the differences reflect mainly enrollment in four-year colleges. Concentrators were more likely to enroll in two-year colleges. Concentrators also were less likely than similar samplers and nonparticipants to complete a bachelor's degree within four to six years. Finally, compared with similar samplers and nonparticipants, concentrators were employed at higher rates in the first five years after high school and had higher earnings.
|WWC 2021012||Math Expressions
This WWC intervention report summarizes the research on Math Expressions, a core curriculum for elementary school students that aims to build students' conceptual understanding of mathematics and to develop fluency in mathematical problem solving and computation. The curriculum encourages student learning of mathematics through real-world situations, visual supports such as drawings and manipulatives, multiple approaches to solving problems, and opportunities for students to explain their mathematical thinking. Based on the research, the WWC found that Math Expressions has no discernible effects on general mathematics achievement for students in grades 1 and 2, relative to other math curricula used in study schools.
|REL 2021087||The Impact of Career and Technical Education on Postsecondary Outcomes in Nebraska and South Dakota
Education leaders in Nebraska and South Dakota partnered with the Regional Educational Laboratory Central to examine how completing a sequence of career and technical education (CTE) courses in high school affects students' rates of on-time high school graduation and their rates of postsecondary education enrollment and completion within two and five years. The study found that CTE concentrators (students who complete a sequence of CTE courses aligned to a specific career field such as manufacturing or education and training) were 7 percentage points more likely than non-CTE concentrators to graduate from high school on time and 10 percentage points more likely to enroll in any type of postsecondary education within two years of their expected high school graduation year. The study also found that CTE concentrators were 3 percentage points more likely than non-CTE concentrators to earn a postsecondary award, such as a professional certificate, diploma, or associate’s or bachelor’s degree, within five years of their expected high school graduation year. CTE concentrators were 4 percentage points more likely than non-CTE concentrators to obtain up to an associate’s degree as their highest postsecondary award within five years of their expected high school graduation year but 1 percentage point less likely to obtain a bachelor’s degree or higher.
|REL 2021083||The Impact of Word Knowledge Instruction on Literacy Outcomes in Grade 5
District leaders in a large urban school district in central Florida wanted to examine the efficacy of a new curriculum designed to enhance the word knowledge of grade 5 students so as to improve reading achievement. The new curriculum, called Word Knowledge Instruction (WKI), consists of 15-minute lessons 4 days a week for 20 weeks. The lessons address state standards and cover 20 prefixes and suffixes. Thirty-nine schools participated in the study, with 92 English language arts (ELA) teachers in high-poverty schools randomly assigned within schools either to use WKI or to continue to use their standard ELA curriculum. Classroom observations revealed that WKI was implemented as intended. WKI had a positive effect, equivalent to an increase of 9 percentile points, on students' ability to correctly extract and spell a base word from a derived word, one of the skills explicitly taught by WKI. WKI had no effect on two other related reading skills that were not directly taught by WKI (students' ability to select a nonword that best fits the grammatical context of a sentence or to use knowledge of word parts to infer meaning of new words) or on students' vocabulary or reading scores. These findings suggest that, although students learned what they were explicitly taught, the transferability to related but not directly taught skills might require more intense or longer duration instruction or additional professional development for teachers.
|REL 2021055||Effects of an Inquiry-Oriented Curriculum and Professional Development Program on Grade 7 Students' Understanding of Statistics and on Statistics Instruction
The purpose of this study was to examine the impact of the Supporting Teacher Enactment of the Probability and Statistics Standards (STEPSS) program on classroom instruction and student understanding of grade 7 statistics. This randomized controlled trial in 40 Broward County, Florida, middle schools compared the STEPSS program condition (a 20-day replacement curriculum unit designed to support teaching and learning of the probability and statistics standards in grade 7, along with four days of professional development for teachers) with practice-as-usual statistics instruction and teacher professional development. The initial study sample included 155 grade 7 mathematics teachers and 14,045 grade 7 mathematics students in the 40 schools. The STEPSS program improved student performance on a test of conceptual understanding of statistics and increased the levels of cognitive demand and classroom discourse in classroom statistics instruction. The magnitude of the effect on student performance was 23 percent of one standard deviation, which is comparable to an increase of 9 percentile points for an average student. In addition, the study found that teachers involved students in tasks involving higher levels of cognitive complexity and engaged their students in higher levels of reasoning and discussion about each other’s ideas regarding probability and statistics in the schools where teachers participated in the STEPSS program. The results of this study suggest that school districts should consider implementing the STEPSS program as a way to improve student understanding of and instructional practice in grade 7 statistics. Mathematics programs that are subjected to randomized controlled trials rarely result in positive impacts on student test performance of the magnitude attained in this study.