Co-Principal Investigators: Pullen, Paige; Aiken, Heather; Leite, Walter; Li, Wei

Related Network Teams: This project is part of the Leveraging Evidence to Accelerate Recovery Nationwide (LEARN) Network, which focuses on adapting and preparing to scale existing, evidence-based products that have the potential to accelerate students' learning relative to pre-pandemic rates of growth for the many learners enrolled in kindergarten through grade 12, postsecondary education, or adult education programs whose learning was affected by the COVID-19 pandemic. The LEARN Network includes the following other projects — Classwide Fraction Intervention with Peer-Assisted Learning Strategies to Support Learning Acceleration (R305N220008), Modifying an Evidence-Based Peer-Mediated Reading Program to Differentiate Instruction and Accelerate Learning Among Underrepresented Groups: Implementing PALS in the Rio Grande Valley (R305N220010), LEARN Network Lead (R305N220012), and Adapting STARI to Accelerate Student Reading Achievement in Middle Grades (R305N220025).

A Research Network involves several teams of researchers who are working together to address a critical education problem or issue. The objective is to encourage information sharing, build new knowledge, and assist policymakers and practitioners to strengthen education policies and programs and improve student education outcomes. The LEARN Network will focus on adapting and scaling up existing, evidence-based products that have the potential to accelerate students' learning for the many learners enrolled in kindergarten through grade 12.

Purpose: The purpose of this project is to adapt and prepare to scale Targeted Reading Instruction (TRI), an evidence-based reading program for kindergarten through grade 3 students. To scale TRI as an effective response to COVID-19, this project will adapt TRI as a digital application ("TRI App") to build foundational reading, oral language, fluency, and comprehension skills of students not yet meeting grade-level benchmarks on school progress monitoring measures. The TRI App will support virtual training and coaching for school-based staff who provide supplemental reading instruction.

Project Activities: In Year 1, the research team will develop the TRI App based on an existing prototype and pilot and refine the TRI App through an iterative development process. In Year 2, the team will evaluate the TRI App using a quasi-experimental design and will conduct market research to determine the best approaches to scaling the TRI App.

Products: Products include a fully developed and operationalized digital application and corresponding professional development for school staff providing supplemental instruction for students reading below grade level. Additionally, the team will produce peer-reviewed publications, reports, and presentations for key school and district stakeholders, policymakers, families, and researchers.

Structured Abstract

Setting: Participating elementary schools include suburban and rural schools in Florida and North Carolina. Schools serving families who are socioeconomically, ethnically, and racially diverse will participate.

Sample: Participants will include 900 kindergarten through grade 3 students reading below grade level and the approximately 50 school staff providing them with supplemental instruction. The network team will identify students through the participating districts' progress monitoring tools as needing supplemental reading instruction.

Intervention: TRI has been evaluated through previous IES-funded randomized controlled trials and is one of only a few reading programs that has shown significant impacts on alphabetics, reading comprehension, and fluency for early elementary students. The core components of TRI include the following: (1) Content (tutors will use TRI activities, strategies, and books to focus on sounds, decoding, word meaning, fluency, and comprehension using real words and texts); (2) Lesson delivery (tutors will structure and pace TRI lessons to guide students efficiently and rapidly through scope and sequence); and (3) Matched instruction (tutors will make instructional decisions diagnostically matched to each individual student based on assessments and TRI performance). In this project, the team will develop the TRI App to provide virtual training and coaching to tutors and to administer TRI lessons. At the initial training, tutors will learn diagnostic strategies for teaching students by watching training videos, observing coach modeling, practicing the activities with one another, and using TRI App activities and strategies with children. A TRI literacy coach will provide virtual coaching sessions to provide real-time feedback as a TRI tutor teaches a lesson using the TRI App. TRI literacy coaches will emphasize tutors' use of data when designing TRI lessons, rapidly building tutor capacity to serve students using data-based decision making.

Research Design and Methods: In Year 1, the research team will iteratively develop and test the TRI App to ensure that it is usable and feasible. In Year 2, the research team will implement a quasi-experimental design to test whether the TRI App improves learning relative to business-as-usual (BAU) instruction and relative to pre-pandemic rates of growth. Schools with tutors using the TRI App will be matched to other schools in the same district implementing BAU instruction. School-level data will be collected from each state and district to create a set of matching variables. Impact measures will be district-administered progress monitoring assessments for kindergarten through grade 3 students and third grade reading tests for third grade students. Fidelity measures will be embedded into the TRI App to assess tutor intervention implementation and coaching. An intent-to-treat analytic approach will estimate treatment effects on student outcomes. Analyses will involve hierarchical linear models to account for the nesting of students and tutors in schools. Additional analyses will compare evaluation students in 2023–24 to pre-pandemic students in 2018-19 to examine growth trajectories for the different cohorts. Market research and corresponding process measures (e.g., focus groups, interviews, surveys) will inform developing, refining, and scaling the TRI App.

Cost Analysis: An embedded cost analysis study will determine the start-up and annual maintenance costs for materials and time/effort of the TRI App. The team will use the ingredients approach to estimate costs such as personnel, training, facilities, materials, and technology.

Related IES Projects: National Research Center on Rural Education Support (R305A040056); The Targeted Reading Intervention: A Web-Based Professional Development Program Targeting K-1 Classroom Teachers and Their Struggling Readers (R305A100654); Investigating the Efficacy of a Web-Based Early Reading Intervention Professional Development Program for K-1 English Learners (R305A160255); An Effectiveness Replication of Targeted Reading Instruction: Investigating Long-Term Student and Teacher Impacts (R305R210007)

Project Websites: https://tri.fpg.unc.edu/targeted-reading-instruction and
https://lastinger.center.ufl.edu/work/literacy/flamingo-literacy/reading-app/