Project Activities
The research team used an RCT to examine the impact of DLL on literacy achievement for three first-grade cohorts of Spanish-speaking ELs who were struggling readers. Researchers randomly assigned students to experimental conditions across 30 schools and collected pre- and posttests of students' Spanish and English reading skills. The team collected measures of program implementation and student and teacher demographic information to examine factors hypothesized to mediate and moderate outcomes. In addition, researchers compared the second- and third-grade literacy achievement of students served by DLL to ELs who were not eligible for DLL (higher achieving) to check for later-emerging effects. The research team collected and analyzed cost data for program costs.
Structured Abstract
Setting
This study included 30 schools in 10 districts across five states (California, Wisconsin, Illinois, Arizona, and Texas).
Sample
The school sample included 30 schools whose collective demographic profile was roughly similar to that of the full population of schools where DLL is implemented. Within schools, teachers were selected if they had already been trained to administer DLL. The treated student sample included the lowest performing ELs in each of three first-grade cohorts, as determined by a series of literacy assessments administered when students began the school year.
Intervention
The DLL program offers daily 30-minute one-on-one instruction for students receiving literacy instruction in Spanish and performing in the bottom 20—25 percent of their class. Trained literacy teachers provide individualized instruction on letter and sound relationships, spelling patterns, comprehension, producing oral language, and increasing vocabularies. DLL lessons are administered as a complement to students' regular classroom bilingual literacy instruction. Each student progresses through DLL at an individualized pace, completing the program in 12—20 weeks. DLL includes teacher training, instructional standards, and ongoing PD including one-on-one support from a DLL teacher leader, a minimum of four PD sessions per year, and multiple classroom observations.
Research design and methods
Researchers implemented a multisite, student- level randomized controlled trial (RCT) in 30 schools across five states. Across the three cohorts, 401 first grade students were randomly assigned to receive DLL services. Approximately half were served immediately during the fall semester, and half received a delayed treatment during the spring semester. The students served in the spring semester served as the control group for those receiving DLL during the fall semester. Baseline tests were given to all students in the fall and posttests after the first semester DLL treatment. Follow-up tests were given in second grade to cohorts 1 and 2, and third grade state standardized test scores were collected for cohort 1. Researchers administered surveys, collected activity logs, and conducted classroom observations to gather information on implementation fidelity. Researchers also conducted interviews and administered surveys to collect cost data on program training and implementation.
Control condition
The study used a delayed treatment design under which the randomly assigned treatment students received DLL during the first semester of first grade and the control students received DLL during the second semester.
Key measures
The researchers used three outcome measures to determine the impact of DLL on literacy achievement: the Iowa Test of Basic Skills (ITBS), the Spanish version of the ITBS (Logramos), and an DLL-specific Spanish-language literacy assessment called the Instrumento de Observación (IdO). The first two assessments are standardized, norm-referenced, and independent from the intervention, whereas the third is specific to DLL and is supported by validity research and evidence from prior studies. Each instrument has three to seven subtests of specific reading skills or practices. In addition to student reading outcomes, data from surveys, activity logs, observations, and interviews were used to quantify implementation fidelity across sites.
Data analytic strategy
Researchers assessed the impact of DLL on literacy outcomes using multilevel regression models with students nested within school sites. These models assessed the intention-to-treat (ITT) impact of the program. The team also examined the effect of DLL on those students who successfully completed the DLL program (treatment on the treated analysis). Student-, teacher-, and school-level predictors were added to the regression models to explore mediating and moderating effects of the program's impact.
Key outcomes
Students receiving DLL outperformed the control students on the Spanish language measures of IdO (significant effect sizes of .36 to .97) and Logramos (significant effect sizes of .20 to .34) with no evidence of cohort differences. DLL did not lead to significant effects on the ITBS English language measure. The treatment on the treated analysis using students who successfully completed the DLL program had similar findings with larger effect sizes for the Spanish language measures. Implementation was found to be uniformly high. Student receipt of more DLL lessons mediated some of the Spanish measures but other aspects of implementation (teacher quality and principal support) did not. The average cost of DLL was $6,047 per student, with a range of $3500 to $9800 by site, 93 percent of which was personnel costs. The cost-effectiveness study found a cost of about $665 to increase a student's total score on IdO by one tenth of a standard deviation and approximately $2,418 to increase a student's total score on Logramos by one tenth of a standard deviation. (Borman et al., 2019; Borman et al., 2023)
People and institutions involved
IES program contact(s)
Products and publications
Publications:
Borman, T. H., Borman, G. D., Houghton, S., Park, S. J., Zhu, B., Martin, A., & Wilkinson-Flicker, S. (2019). Addressing literacy needs of struggling Spanish-speaking first graders: First-year results from a national randomized controlled trial of Descubriendo la Lectura. AERA Open, 5(3).
Borman, T. H., Borman, G. D., Park, S. J., Zhu, B., & Houghton, S. (2023). The Spanish-and English-Language Literacy Impacts of Descubriendo la Lectura across Three Experimental Replications. Journal of Research on Educational Effectiveness, 1-30.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
