When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program New Findings
Structured Abstract
Citation: James-Burdumy, Susanne, Mark Dynarski, Mary Moore, John Deke, Wendy Mansfield, and Carol Pistorino. "When Schools Stay Open Late: The National Evaluation of the 21st Century Community Learning Centers Program: Final Report." U.S. Department of Education, National Center for Education Evaluation and Regional Assistance.
Background: The 21st Century Community Learning Centers program has supported after-school programs since 1998. Research on the effects of after-school programs has been inconclusive, leading to an ongoing debate about the effects of after-school programs.
Purpose: To examine the implementation of the 21st Century Community Learning Centers after-school program and assess its impacts on students. Earlier reports from this study presented findings based on two school years of data for middle school students and one school year of data for elementary school students. Key impact findings from the first report include no improvement in homework completion, limited effects on academic outcomes, no reduction in self-care, no improvements in safety and behavior, higher levels of parental involvement for the treatment group relative to the control group, and few effects on developmental outcomes. Key impact findings from the second report include higher levels of supervision by adults for treatment-group students relative to control-group students, lower levels of supervision by siblings for treatment-group students relative to control-group students, no reduction in self-care, few impacts on academic outcomes, improved feelings of safety after school for elementary students in the treatment group relative to students in the control group, mixed evidence on negative behavior for middle school students, some impacts on parents of elementary students, and few impacts on developmental outcomes. The purpose of the current report is to present impact analyses based on two years of follow-up data for elementary students.
Setting Twenty-six 21st Century centers in 12 school districts.
Subjects: A total of 2,308 elementary students eligible for and interested in attending a 21st Century Community Learning Center. A total of 973 students applied to 18 centers in fall 2000, and 1,335 applied to 8 centers in fall 2001.
Intervention: 21st Century centers typically offered homework sessions, academic activities, enrichment activities, such as art, drama, or music, and recreation activities.
Research Design: Randomized controlled field trial. Students were randomly assigned either to the 21st Century center group (1,258 students) or to a control group (1,050 students).
Control or Comparison Condition: Control students could participate in any other after-school activities and programs to which they were entitled or eligible, but they were not eligible to participate in 21st Century after-school centers for two years.
Data Collection and Analysis: Data on students' supervision after school, academic achievement, behavior, developmental outcomes, and feelings of safety after school were collected from parents, teachers, students, and school records in fall 2000 (baseline), spring 2001 (first followup), and spring 2002 (second followup) for the first cohort of students, and one year later for students who applied to centers in fall 2001. The Stanford Achievement Test in reading was administered at baseline and followup. Regression-adjusted impact estimates that compare the outcomes of treatment and control students were calculated to assess differences between the 21st Century and control groups. We also collected implementation data from program staff and principals and conducted two visits to each site, once during each of the two years of the study.
Findings: Earlier analyses found few impacts of 21st Century programs. It was hypothesized that an additional year of follow-up data might show positive effects because students had the opportunity to participate for a second school year, and change in some outcomes might require more time than others. Analyses of an additional year of follow-up data do not yield support for this hypothesis. Treatment-group students were less likely than control-group students to be in parent care and more likely to be in the care of other adults, but they were no less likely than control-group students to be in self-care. Treatment-group students did not have higher levels of academic achievement as measured by reading test scores or grades in math, science, social studies, and English relative to control-group students. There was evidence of higher levels of negative behavior among the treatment group relative to the control group on multiple outcomes, including suspensions, teachers calling students' parents about behavior, and students being disciplined by teachers. There were mixed effects on developmental outcomes. Treatment-group students had improved feelings of safety after school relative to control-group students.
Conclusions: This study finds that elementary students who were randomly assigned to attend the 21st Century Community Learning Centers after-school program were more likely to feel safe after school, no more likely to have higher academic achievement, no less likely to be in self-care, more likely to engage in some negative behaviors, and experience mixed effects on developmental outcomes relative to students who were not randomly assigned to attend the centers.