Scaling up First Things First: The challenge of scaling up educational reform.

Findings

Staying in School outcomes—Indeterminate effect found for the domain

Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement     index	Evidence tier
One-year persistence rates	First Things First vs. None	Posttest	Ninth graders; 14 students	76.30	77.30	No	--

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Study Details

Setting

First Things First was implemented in three high schools in the Houston Independent School District: Lee High School, Sam Houston High School, and Sharpstown High School. During the study period, these three high schools served student populations that were primarily Hispanic. Lee High School began implementing First Things First in the fall of the 2001/02 school year, and Sam Houston and Sharpstown began implementing the program the following school year, 2002/03.

Study sample

The study examined the performance of students attending schools in the Houston Independent School District. The main analysis sample included students from three high schools and four middle schools implementing First Things First, as well as students from a set of matched comparison schools. The middle school study did not examine outcomes relevant to WWC dropout prevention reviews. Therefore, the results in this intervention report pertain only to the high school analysis. The high school sample consists of all students enrolled in the study schools during the study period: three years prior to First Things First implementation and one year after implementation. This included a total of 7,891 high school students in the First Things First schools. The study authors did not report the number of students in comparison high schools. Comparison schools in the Houston school district were matched to each First Things First high school based on overall student performance at the schools on standardized achievement tests during the baseline period. High schools in the district whose mean combined reading and math scores on the Stanford Achievement Test, ninth edition (SAT-9), were within 0.25 standard deviations of the mean for the First Things First school were selected as comparison schools. This methodology resulted in 10 comparison schools for 2 of the First Things First high schools and 11 comparison schools for the other. A high school could be chosen as the comparison school for more than one First Things First school, and this often occurred. A total of 13 Houston high schools served as comparison schools for the First Things First Houston study. The high school study examined two cohorts of students. Cohort 1 consisted of one intervention high school that implemented First Things First in 2001 and its matched comparison schools. Cohort 2 consisted of two intervention high schools that began implementing First Things First in 2002 and their comparison schools. The WWC used results for cohorts 1 and 2 combined to rate the effectiveness of First Things First. In the years prior to the implementation of First Things First, the three First Things First schools and their comparison high schools had, on average, similar attendance and promotion rates and served students who had similar tenth-grade passing rates on math and reading tests. The study authors indicated that the First Things First high schools in Houston were heavily Hispanic, while other low-achieving high schools in Houston were heavily African-American. Therefore, it was not possible to match high schools closely on both their racial and ethnic composition and their student performance.

Intervention Group

First Things First targeted three high schools in Houston with low-achievement levels. Each of these schools implemented the key features of the three components of the whole school reform: Small learning communities. First Things First reorganizes schools into small learning communities of up to 350 students and their teachers. The small learning communities in Houston were focused around a central theme (for example, technology), and students remained in the same communities throughout high school. First Things First also recommends that schools reduce student-teacher ratios in math and language arts classes to increase the amount of individualized attention that students receive. Class sizes in the Houston study schools were reduced from an average of 26 students to 20 students between the planning year and the second implementation year. Family and student advocate system. Advocates met one-on-one with students weekly and contacted the students’ families at least monthly to discuss their academic and personal progress. In at least one Houston high school the advocates became counselors for the students, helping them to schedule the classes they needed to progress toward graduation. Consistent with the model’s guidelines, advocates in Houston high schools met weekly with their students as a group. These group sessions were held as either a daily “homeroom” period or a weekly class meeting. Instructional changes and supports. First Things First contracted with Kagan Cooperative Learning, Inc., to train teachers on cooperative learning strategies intended to complement the small learning communities. It also called for increased instructional time for math and language arts courses.

Comparison Group

Matched comparison schools were Houston high schools that did not implement First Things First. The study identified comparison schools from the districts that were similar in average performance on standardized achievement tests in the three years preceding program implementation. High schools in the district whose mean scores on the Stanford Achievement Test, ninth edition (SAT-9), for reading and math combined were within 0.25 standard deviations of the mean for the First Things First school were selected as comparison schools.

Outcome descriptions

The outcome of interest from this study was a measure of the percentage of ninth graders who attended at least one day of school during the following year, referred to as the “one-year persistence rate.” The study also examined First Things First’s effects on attendance and standardized test scores. These outcomes do not fall within the three domains examined by the WWC’s review of dropout prevention interventions (staying in school, progressing in school, completing school) and are not included in this report.

Support for implementation

Teachers at First Things First high schools were regular teachers employed by the Houston Independent School District. As part of the instructional changes, teachers in First Things First schools were trained in cooperative learning strategies by Kagan Cooperative Learning. Each school employed a First Things First director to serve as a liaison between the district and the First Things First schools. Staff from IRRE provided ongoing professional development for teachers in the implementation of the family advocate system.

Additional Sources