The study took place in five districts in four states in the western, southern, and northeastern United States. Most districts were located in mid-size to large cities.
The study used a cluster randomized controlled trial design. Thirty-seven schools that met the study eligibility criteria were randomly assigned to intervention or comparison groups in spring 2011 after blocking by school district. To be eligible to participate in the study, schools were required to serve grades K–5, have at least 40% of their students eligible for free or reduced-price lunch, and be willing to participate in the study and support program implementation. The program was implemented for all students in the schools starting in fall 2011.
The authors used three samples to evaluate the effectiveness of SFA®, which they refer to as the main sample, the spring sample, and the auxiliary sample. The main sample focused on students who were present in schools at the time of baseline and outcome assessments. The spring sample included all students who had at least one valid score on the end-of-year outcomes. The auxiliary sample consisted of students who were present in grades 3, 4, or 5 in the study schools during program implementation years. All three samples may include students who moved into the study schools after random assignment.
For the effectiveness ratings, the WWC focused on third-year findings from the sample of students who had at least one valid score on the end-of-year outcomes (referred to as the spring sample in the study). The third-year analyses focused on second-grade students who were in kindergarten when implementation began. This cohort included 1,557 students in 19 SFA® schools and 1,350 students in 18 comparison schools.
Across all study schools, 57% of the population received free or reduced-price lunch, 62% of students were Hispanic, 23% were Black, and 14% were White. Males made up 52% of the overall school sample.
Intervention students received features of the full SFA® program, including the SFA® reading curriculum that is the focus of this intervention report, tutoring for students in grades 1–3, a facilitator who worked with school personnel, and training for all intervention teachers. Some other features of the full SFA® program, such as regular tutoring for struggling students, periodic testing and regrouping, and support for families, were not provided to all students in all schools. The study relied on local district coaches rather than coaches employed by SFA®. The SFA® model calls for a 90-minute reading block each day, and most schools adhered to this. Schools began using the program for the first time at the beginning of the first study year, and in general improved their implementation over the course of the study based on the authors’ monitoring.
The comparison condition included schools that implemented standard reading programs from publishers such as Macmillan/McGraw-Hill, Houghton Mifflin Harcourt, and Scott Foresman. During the 3-year study period, most comparison schools continued to use the same curriculum, while others switched from one common program to another.
Support for implementation
Each school implementing SFA® appointed a facilitator who oversaw the implementation of the program. Principals and other school leaders attended a week-long conference the summer before implementation, in which they were introduced to the various parts of the programs. SFA® coordinators visited the schools for 4 days before the beginning of the school year. One day of programming focused on principals and school leaders, the second day on all teachers, and the third and fourth days on reading teachers. During the school year, SFA® coaches visited the schools implementing the program to provide additional support. This was focused primarily on assisting principals and other leaders in implementing program features, but also included classroom visits and feedback on lessons.