|Title:||WORDS (Workshop on Reading Development Strategies) for Pandemic Recovery in Nebraska|
|Principal Investigator:||Hebert, Michael||Awardee:||University of California, Irvine|
|Program:||Research to Accelerate Pandemic Recovery in Special Education [Program Details]|
|Award Period:||4 years (07/01/2022 – 06/30/2026)||Award Amount:||$2,682,360|
|Type:||Initial Efficacy||Award Number:||324X220115|
Previous Award Number: 324X220111
Education Agency Partner: Nebraska Department of Education
Co-Principal Investigator: Goodrich, J. Marc
Purpose: The purpose of this project is to evaluate Workshop on Reading Development Strategies (WORDS), a comprehensive professional development (PD) program for teachers designed to support implementation of Tier 2 intensive interventions in reading for students with or at risk for reading disabilities in grades K–3. Many Nebraska schools have indicated that students with reading difficulties, especially those with cognitive or intellectual disabilities, have been disproportionately impacted by the COVID-19 pandemic as they have not had access to typical reading instruction and supports. In response to this issue, this project is designed to support educators in providing high-quality reading instruction to accelerate growth in reading for students showing a need for more intensive reading intervention. Specifically, this project aims to implement WORDS with students falling below the state identified cut-scores on Nebraska reading screeners; evaluate its impact on student reading outcomes, including for students with cognitive or intellectual disabilities specifically; and foster sustainability of WORDS within schools for future years.
Project Activities: The research team will evaluate the impact of WORDS on student outcomes using two quasi-experimental designs—regression discontinuity and propensity score matching. Teachers within WORDS schools will receive practice-based PD workshops for improving reading intervention implementation as well as instructional observations and coaching supports, implement Tier 2 reading instruction and after-school tutoring for all K–3 students in their school who fall below established screener cut scores, and receive leadership training to support the sustainability of WORDS.
Products: The primary product of this project will be an understanding of the effects and costs of the WORDS program on the reading outcomes of students in grades K–3 with or at risk for reading disabilities. Products will also include peer-reviewed publications and presentations, a publicly available dataset, and additional products that reach stakeholders such as practitioners and policymakers, including the Nebraska Department of Education.
Setting: The research will take place in elementary schools in rural areas of Nebraska.
Sample: For the regression discontinuity studies, the sample will include approximately 1700 students and 96 teaching staff (general and special education teachers and paraprofessionals) in grades K–3 across 12 elementary schools. Although data for all students in grades K–3 will be analyzed, it is estimated that roughly 20% of these students will score below established criteria on state-approved reading assessments and therefore be identified as with or at risk for reading disabilities and in need of Tier 2 reading instruction. For the propensity score matching studies, only students scoring below the established criteria will be included, which will include approximately 340 students (20%) from the original 12 regression discontinuity schools as well as an additional 340 student who qualify from the 12 matched control schools.
Intervention: WORDS is a multi-component program designed to address reading difficulties at multiple levels of the reading instructional “system.” That is, WORDS supports student reading through (a) enhancing general classroom reading instruction, (b) improving alignment between assessment and instruction, (c) implementing evidence-based reading intervention, (d) increasing instructional opportunities for students, and (e) developing reading leadership teams within local school districts. Teachers will receive PD along with coaching and feedback to improve their implementation of high-quality Tier 2 reading instruction. The reading interventions implemented within each school will include any reading intervention that meets Level 2 of the ESSA Levels of Evidence for Interventions.
Research Design and Methods: WORDS will be evaluated with a series of quasi-experimental studies using regression discontinuity design (RDD) and a propensity score matching control group design. Two RDD studies will be run, one from fall to winter and the next from winter to spring, within each of the 4 project years, for a total of eight RDD studies. Students falling below a state mandated cut-score on the MAP Growth assessment will be assigned to receive intervention. Performance of students falling below the cut-score will be compared to that of students performing above the cut-score to evaluate intervention efficacy. At the end of each school year, the 12 WORDS schools that participated in the RDD studies will be matched through a propensity score matching design to 12 additional schools that did not receive WORDS to examine reading outcomes of students who performed below the cut score on the MAP Growth assessment. For these studies, reading outcomes will be assessed during the school year both immediately following intervention and at the end of each school year for up to 3 years following intervention starting in grade 3. In addition, teachers will be surveyed to gather their perceptions of participating in WORDS, how their instruction was impacted, and impacts on their students. Further descriptive data will be gathered to learn about the experience of students with cognitive and intellectual disabilities who participated in WORDS.
Control Condition: For the regression discontinuity design studies, all students who do not fall below the cut-score on the MAP Growth assessment for WORDS schools will serve as the control group. For the propensity score matching study, students performing below the cut-score on the MAP Growth assessment in the matched schools (which did not participate in WORDS) will serve as the control group.
Key Measures: This project will include three key outcome assessments that are administered across all participating Nebraska school districts: MAP Growth assessment (which will also be used to identify students qualifying for Tier 2 instruction and the propensity score matching), DIBELS 8, and end-of-year state reading assessments (Nebraska Student-Centered Assessment System [NSCAS]). MAP Growth and DIBELS 8 will be administered in fall, winter, and spring each year for kindergarten through third grade students. NSCAS will be administered for longitudinal analyses in spring of each year when students reach grades 3 through 6. Fidelity of WORDS PD will be measured with checklists. Fidelity of teacher implementation with students will be checked and supported through observations conducted six times each year. In addition, surveys will be used to collect student and teacher demographic information, as well as teacher training and experience.
Data Analytic Strategy: Researchers will use linear mixed modeling to examine the impacts of WORDS from the RDD studies. They will also use multi-level modeling to examine the impacts from the school-level propensity score matching design to control for school- and teacher-level variances.
Cost Analysis: The ingredients method will be used to determine the cost of the resources needed to implement WORDS. A cost analysis will be performed to estimate the total and per student cost of implementing WORDS.