Limitations and Conclusions

The EDS used a random assignment design, the strongest approach for estimating program impacts. However, projects volunteered for this study instead of being randomly selected, so we cannot generalize to the Even Start population on a strict statistical basis. EDS families are more likely than the population of Even Start families to be Hispanic, and EDS projects are more likely than the population of Even Start projects to be in urban areas. Thus, findings from the EDS are most relevant to urban projects that serve large numbers of Hispanic/ESL families. In addition, to be included in the analysis for this study children and parents were required to have a complete set of data for a given outcome variable (i.e., data at pretest, posttest, and follow-up) with all direct assessments administered in English. This limits generalizability to families that are relatively stable over a two-year period as well as children and parents who were comfortable enough with English to be assessed in that language.

As was the case at posttest, Even Start children and parents made gains on a variety of literacy assessments and other measures at follow-up, but they did not gain more than children and parents in the control group. It had been hypothesized that follow-up data might show positive effects because (1) Even Start families had the opportunity to participate for a second school year, and (2) change in some outcomes might require more time than others. However, the follow-up data do not support either of these hypotheses.

Several reasons have been advanced for the lack of documented effectiveness of Even Start family literacy projects (St.Pierre, et al, 2003). Some of these include that instructional services may not be sufficiently intensive, the quality of instructional services may be insufficient, and that families do not participate enough.