Ask a REL Response

Best practices, summer school — June 2018

Question

What does the research say about best practices for summer school?

Response

Following an established REL West research protocol, we conducted a search for research reports and resources on best practices for summer school. The sources included ERIC, Google Scholar, and PsychInfo. (For details, please see the methods section at the end of this memo.)

We have not evaluated the quality of references and the resources provided in this response. We offer them only for your reference. Also, we searched for references through the most commonly used sources of research, but the list is not comprehensive and other relevant references and resources may exist. References are listed in alphabetical order, not necessarily in order of relevance.

Research References

Beckett, M., Borman, G., Capizzano, J., Parsley, D., Ross, S., Schirm, A., & Taylor, J. (2009). Structuring out-of-school time to improve academic achievement: A practice guide (NCEE #2009-012). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://ies.ed.gov/ncee/wwc/PracticeGuide/10

From the abstract: “Out-of-school time programs can enhance academic achievement by helping students learn outside the classroom. The purpose of this practice guide is to provide recommendations for organizing and delivering school-based out-of-school time (OST) programs to improve the academic achievement of student participants. The five recommendations in this guide are intended to help district and school administrators, out-of-school program providers, and educators design out-of-school time programs that will increase learning for students. These recommendations are: (1) Align the OST program academically with the school day; (2) Maximize student participation and attendance; (3) Adapt instruction to individual and small group needs; (4) Provide engaging learning experiences; and (5) Assess program performance and use the results to improve the quality of the program. The guide also describes the research supporting each recommendation, how to carry out each recommendation, and how to address roadblocks that might arise in implementing them. The scope of this practice guide is limited to programs that (1) serve elementary and middle school students; (2) are organized by or conducted in partnership with a school or school district; and (3) aim to improve academic outcomes.”

Capizzano, J., Bischoff, K., Woodroffe, N., & Chaplin, D. (2007). Ingredients of a successful summer learning program: A case study of the Building Educated Leaders for Life (BELL) accelerated learning summer program. Princeton, NJ: Mathematica Policy Research. Retrieved from https://eric.ed.gov/?id=ED497332

From the abstract: “Based on positive results from a previous evaluation of a summer learning intervention, the current report describes the specific elements of the successful program so it can be replicated, and investigates potential barriers to implementation and replication. The study estimated impacts of the program overall; the authors could not identify which elements caused the positive impacts other than to say that the elements described were present in this successful program and thus of potential importance. This observation and interview-based process study describes activities that occurred during the summers of 2004 and 2005 in an academically-oriented summer program that receives both federal and private funding: the Building Educated Leaders for Life (BELL) Accelerated Learning Summer Program. The BELL program is a program with an asset-based youth development approach. It employs well-developed curricula in both reading and math and contains features of positive developmental settings outlined by the National Research Council and Institute of Medicine. The program has demonstrated effectiveness in reducing summer learning loss among low-income children based on both random assignment external evaluation and two internal evaluations. The current report allows researchers to examine whether BELL’s program components are implemented with fidelity, allowing the research team to better understand the nature of the BELL intervention in practice and assist with the interpretation of outcome study results. The report also describes implementation issues that may affect whether the BELL program can be replicated in other sites, such as obtaining sustainable funding, high-quality program staff and physical space. In addition, the authors discuss unique characteristics associated with the existing BELL sites that may not be easily reproduced, such as an abundance of partners in the community from which to draw resources, a lack of competing programs, or other environmental characteristics that help to support the program. This process report was developed as part of a larger study that included a random assignment impact evaluation of the summer program. Students were randomly assigned to treatment and control groups. Only the treatment group was given access to the BELL summer program. Impacts were estimated by comparing outcomes for the treatment and control groups. Random assignment was done in both the summers of 2004 and 2005. However, student survey response rates were very low in 2004 so the impact report cited is based on the 2005 data.”

McCombs, J. S., Augustine, C., Schwartz, H., Bodilly, S., McInnis, B., Lichter, D., & Cross, A. B. (2012). Making summer count: How summer programs can boost children’s learning. Education Digest: Essential Readings Condensed For Quick Review, 77(6), 47–52. Retrieved from https://www.rand.org/content/dam/rand/pubs/monographs/2011/RAND_MG1120.pdf

From the abstract: “During summer vacation, many students lose knowledge and skills. By the end of summer, students perform, on average, one month behind where they left off in the spring. Participation in summer learning programs should mitigate learning loss and could even produce achievement gains. Indeed, educators and policymakers increasingly promote summer learning as a key strategy to improve the achievement of low-performing students. Rigorous studies have shown that strong summer programs can achieve several important goals: (1) reverse summer learning loss; (2) achieve learning gains; and (3) give low-performing students the chance to master material that they did not learn during the school year. The authors recommend that districts and other providers invest in staffing and planning for summer learning programs, actively incorporate practices that will help ensure the success of programs, and maximize the benefits of partnerships and a variety of funding sources. They also offer recommendations for policymakers and funders who are interested in supporting summer learning programs.”

Snipes, J., Huang, C., Jaquet, K., & Finkelstein, N. (2015). The effects of the Elevate Math summer program on math achievement and algebra readiness (REL 2015–096). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. Retrieved from https://ies.ed.gov/ncee/edlabs/regions/west/pdf/REL_2015096.pdf

From the abstract: “To raise math success rates in middle school, many schools and districts have implemented summer math programs designed to improve student preparation for algebra content in grade 8. However, little is known about the effectiveness of these programs. While students who participate typically experience learning gains, there is little rigorous evidence evaluating the effects of the programs on math achievement or readiness for algebra content. This study fills that void by rigorously examining the effects of one such summer program (the Elevate summer math program) on student achievement. In summer 2014, the Silicon Valley Education Foundation (SVEF), the research team, and several Silicon Valley school districts collaborated on a randomized controlled trial to assess the effects of the Elevate Math summer program on math achievement, algebra readiness, and attitudes toward math. The study focused on three primary questions: (1) What is the impact of the Elevate Math summer program on the math achievement and algebra readiness of rising grade 8 students?; (2) What is the impact of the Elevate Math summer program on math achievement in the math topic areas most closely aligned with the program’s curriculum?; and (3) What is the impact of the Elevate Math summer program on the math interest and math self-efficacy of rising grade 8 students? The randomized controlled trial was conducted in summer 2014 at eight schools in six districts in California’s Silicon Valley. Participating districts identified eligible students based on existing grade 6 California Standards Test (CST) data. The districts’ enrollments range from 2,487 to 13,162, with an average of 9,426. The percentage of English learner students in each district ranges from 19% to 53%, with an average of 38%. Students were randomly assigned to a treatment group that received access to the program at the beginning of the summer or to a control group that received access to the program later in the summer. Math achievement was measured using the Mathematics Diagnostic Testing Project (MDTP) Algebra Readiness test, which was administered to the treatment and control groups on the first and last days of their participation in the summer program. The Elevate Math summer program significantly improved math achievement and algebra readiness among participating grade 7 students. The program improved the math achievement of the treatment group compared with the control group across several metrics. The Elevate Math summer program also had a positive, statistically significant effect on algebra readiness. Despite the Elevate Math summer program’s effects, students’ math achievement at the end of the program suggested that many students were still not ready for the algebra content in grade 8 math courses. There were no significant impacts on math interest or math self-efficacy. The estimated level of interest in math for the treatment group was higher than that of the control group, but the difference was not statistically significant.”

Additional Organizations to Consult

California School Boards Association (CSBA) – https://www.csba.org

From the website: “CSBA is the nonprofit education association representing the elected officials who govern public school districts and county offices of education. With a membership of nearly 1,000 educational agencies statewide, CSBA brings together school governing boards, and administrators from districts and county offices of education to advocate for effective policies that advance the education and well-being of the state’s more than 6 million school-age children. A membership-driven association, CSBA provides policy resources and training to members, and represents the statewide interests of public education through legal, political legislative, community and media advocacy.”

REL West note: One report published by CSBA in 2013 is relevant to this request, as follows:

California School Boards Association. (2013). What constitutes an effective summer program? West Sacramento, CA: Author. Retrieved from https://www.csba.org/productsandservices/allservices/~/media/f5f07e67c1e94e6fb8f0c323922bb66e.ashx

Excerpt: “Regardless of the program focus, successful programs share common characteristics. The Summer Matters campaign has piloted summer learning programs in 13 districts across the state and, based on the National Summer Learning Association’s quality standards, has identified some core elements of a high-quality program: (1) Children are engaged in learning activities that are active and meaningful, promote collaboration, expand their horizons and build mastery in a safe environment; (2) Skilled staff have strong, positive relationships with students and are intentionally working to deliver engaging learning experiences that meet students’ emotional, social and academic needs and goals; (3) The program is managed by visionary, knowledgeable leaders, including school board members and superintendents, committed to continuously improving program quality and securing the resources needed to deliver the learning experiences students need; (4) The program is anchored in its community, with tangible support from families, community-based organizations and civic leaders partnering with schools to maximize resources and provide the best overall experience for youth; and (5) Five to six weeks of full-day programming address the needs of children and their families. These elements are consistent with what researchers from the RAND Corp. found in their much-cited report Making Summer Count.”

Child Care & Early Education Research Connections – https://www.researchconnections.org

From the website: “Child Care & Early Education Research Connections, launched in 2004, promotes high quality research in child care and early education and the use of that research in policy making. Our vision is that children are well cared for and have rich learning experiences, and their families are supported and able to work. Through this website, we offer research and data resources for researchers, policy makers, practitioners, and others. Research Connections is a partnership between the National Center for Children in Poverty (NCCP) at the Mailman School of Public Health, Columbia University; the Inter-university Consortium for Political and Social Research (ICPSR) at the Institute for Social Research, the University of Michigan.”

REL West note: One report published by Child Care & Early Education Research Connections in 2016 is relevant to this request, as follows:

Stephens, S. A. (2016). Community-based summer learning programs for school-age children: Research-to-policy resources. Washington, DC: Child Care & Early Education Research Connections. Retrieved from https://www.researchconnections.org/childcare/resources/32136/pdf

From the abstract: “Summer learning experiences for school-age children can be provided in a variety of ways and settings, including summer school programs (often remedial), community-based programs (often a continuation of afterschool programs), and home-based programs (in which families are provided with information and resources to encourage reading, often run by libraries). Research has indicated that all three types of summer learning programs can have a positive impact on children’s retention of key skills when these programs are of high quality, recruit vulnerable children, and engage families in ensuring consistent attendance. Having an impact on a broad scale requires that policies and infrastructure, including funding, are in place to expand effective programs. This Research-to-Policy Resource List compiles publications and documents published in 2010 and later on the following topics related to community-based summer learning programs for school-age children: Effective curricula, best practices and exemplary models; and State policy options.”

Child Trends – https://www.childtrends.org

From the website: “Child Trends is a nonprofit, nonpartisan research center that studies children at all stages of development. We seek to improve the lives of children and youth by conducting high-quality research and sharing it with the people and institutions whose decisions and actions affect children. Founded in 1979, Child Trends helps keep the nation focused on children and their needs by identifying emerging issues; evaluating important programs and policies; and providing data-driven, evidence-based guidance on policy and practice.”

REL West note: One report published by Child Trends in 2009 is relevant to this request, as follows:

Terzian, M., & Moore, K. A.(2009). What works for summer learning programs for low-income children and youth: Preliminary lessons from experimental evaluations of social interventions. Washington, DC: Child Trends. Retrieved from http://www.wallacefoundation.org/knowledge-center/Documents/Effective-and-Promising-Summer-Learning-Programs-Fact-Sheet.pdf

From the abstract: “Children and youth who reside in economically disadvantaged households and in low-resource, urban neighborhoods are more likely to lose ground in math and reading over the summer than their higher-income peers. Although summer learning programs are a promising strategy for narrowing this achievement gap, surveys indicate that only 25 to 36 percent of U.S. children between 6 and 11 years of age attend summer programs (excluding summer school). This Fact Sheet presents some emerging lessons from 11 summer learning programs that were evaluated using experimental research designs. These programs are included in the Child Trends’ online database of experimentally-evaluated, out-of-school time interventions—LINKS (Lifecourse Interventions to Nurture Kids Successfully). All of the programs were implemented with economically disadvantaged children and youth. The findings of this synthesis suggest that summer learning programs can be effective and are likely to have positive impacts when they engage students in learning activities that are hands-on, enjoyable, and have real-world applications. This review also suggests some insights into promising practices. Programs that are guided by grade-level curricular standards, are led by experienced teachers, conduct classes with 15 or fewer students and at least two adults, and complement group learning with individual support were also found to be effective.”

The National Summer Learning Association (NSLA) – https://www.summerlearning.org

From the website: “The National Summer Learning Association serves as a network hub for thousands of summer learning program providers and stakeholders across the country, providing tools, resources, and expertise to improve program quality, generate support, and increase youth access and participation. We offer professional development, quality assessment and evaluation, best practices dissemination and collaboration, and strategic consulting to states, school districts, community organizations, and funders.”

REL West note: One report published by NSLA in 2018 is relevant to this request, as follows:

National Summer Learning Association. (2018). The New Vision for Summer School Network. Baltimore, MD: Author. Retrieved from https://www.summerlearning.org/wp-content/uploads/pdf/NVSS-Flyer-8.2018.pdf

From the abstract: “The New Vision for Summer School Network (NVSS Network) is an affinity group of school district members and national partners committed to a broad vision for summer learning—one that engages more children and youth, boosts academic achievement, and influences teaching and learning throughout the year. As a network, we envision a future where school districts and schools include summer learning and enrichment as part of a 12-month plan for learning. Together with their community partners, districts will close this critical opportunity gap and ensure that all young people have access to high-quality summer learning experiences that help them succeed in college, career, and life. To accomplish its vision, the NVSS Network is organized by five principles and underlying strategies to strengthen and expand summer learning programs: (1) Increase and enhance the scope of traditional summer school; (2) Target participation by students who would benefit the most; (3) Strengthen systems-level supports through community-wide partnerships and coordination; (4) Provide innovative professional development for staff; and (5) Embed summer learning in to the district’s school-year operations.”

RAND Corporation – https://www.rand.org

From the website: “The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND’s research and analysis address issues that impact people around the world including security, health, education, sustainability, growth, and development. Much of this research is carried out on behalf of public and private grantors and clients.”

REL West note: One report published by RAND in 2013 is relevant to this request, as follows:

Augustine, C. H., McCombs, J. S., Schwartz, H. L., & Zakaras, L. (2013). Getting to work on summer learning: Recommended practices for success. Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/research_reports/RR300/RR366/RAND_RR366.pdf

From the preface: “This report offers guidance to district leaders across the country who are interested in launching summer learning programs or improving programs that are already established. Our recommendations are based on the evaluations of summer programs in six urban districts in the summer of 2011. These districts—Boston; Cincinnati; Dallas; Duval County, Florida; Pittsburgh; and Rochester, New York—were selected for a multiyear demonstration project funded by The Wallace Foundation to assess their effectiveness in improving student achievement. They are among the nation’s most advanced in their experience with comprehensive, voluntary summer learning programs.”

The Wallace Foundation – http://www.wallacefoundation.org

From the website: “The Wallace Foundation is a national philanthropy that seeks to improve education and enrichment for disadvantaged children. The foundation has an unusual approach: funding projects to test innovative ideas for solving important social problems, conducting research to find out what works and what doesn’t and to fill key knowledge gaps—and then communicating the results to help others.”

REL West note: Two reports published by the Wallace Foundation in 2009 are relevant to this request, as follows:

McLaughlin, B., & Pitcock, S. (2009). Building quality in summer learning programs: Approaches and recommendations. New York, NY: Wallace Foundation. Retrieved from http://www.wallacefoundation.org/knowledge-center/documents/building-quality-in-summer-learning-programs.pdf

From the abstract: “As a field, summer programs vary widely on a number of dimensions, including the settings in which they take place, the operators of the programs, the content or focus of the activities and the target population. Although diversity can be viewed positively because it allows families to individualize the summer experiences of their children, it also has some drawbacks. Most significantly, the range in summer options often means a deep inequality in the daily experiences of higher- and lower-income youth. While some youth spend the summer studying abroad entrenched in cultural learning experiences, others may pass the time watching television or simply hanging out. For low-income, urban families, cost and proximity are frequently cited as primary considerations in selecting a summer program for their children. These considerations often narrow the realistic options for summer programming. In addition, information about summer programs is rarely aggregated at the community level, making programs difficult to find. Four types of operators: schools, parks and recreation agencies, child care centers and community-based and faith-based organizations, typically offer summer programming targeted to disadvantaged youth. Among these four operator types, access to quality supports—for curriculum, staffing, standards of practice and tools to assess quality—varies based on conditions of funding, the focus of the program, and whether or not the program is connected to an intermediary or national umbrella organization…….The authors recommend several action steps to help program operators access resources and achieve quality programming: (1) Adapt out-of-school time curriculum for summer; (2) Identify and validate baseline quality standards for summer; (3) Promote and disseminate quality assessment tools specific to summer; (4) Connect summer programs to intermediaries; (5) Develop an online clearinghouse of quality supports for summer programming; (6) Professionalize staff in the field of out-of-school time and summer learning; and (7) Communicate a new vision for summer school.”

Terzian, M., Moore, K. A., & Hamilton, K. (2009). Effective and promising summer learning programs and approaches for economically-disadvantaged children and youth: A White Paper for the Wallace Foundation. New York, NY: Wallace Foundation. Retrieved from http://www.wallacefoundation.org/knowledge-center/documents/effective-and-promising-summer-learning-programs.pdf

From the abstract: “This White Paper summarizes findings from an extensive literature review that was conducted to identify the most promising models and approaches for meeting the needs of low-income children, youth, and families during the summer months. Special attention is paid to summer learning programs that serve diverse, urban low-income children and youth. Data on program participation suggest that children and youth who would stand to benefit the most from summer learning programs are the least likely to participate. This paper focuses on summer learning programs, as opposed to recreational, wilderness, or child care programs. Summer schools that focus on remediation are also not reviewed.  Five types of summer learning programs are reviewed: (1) Educational and Cognitive; (2) Youth Development; (3) Career Development; (4) Health and Fitness; and (5) Multi-element. Experimental and non-experimental studies, as well as informal evaluation reports and papers reporting practitioner insights, were reviewed to identify effective and promising summer learning practices. Program impacts from experimental evaluations were identified for outcomes ranging from math and reading achievement to an increased likelihood of employment. Drawing from a limited number of ten experimental evaluations, we found that reading achievement gains were achieved for a handful of programs, whereas math achievement was less often a program focus and impacts were less consistent. Few impacts were found on high school completion, college enrollment, and employment. Finally, a lack of evidence was found for youth development and health and fitness outcomes due to the fact that these outcomes were rarely, if ever, evaluated. Several recommendations for practice, research, and evaluation are presented. The literature reviewed, though limited, indicates that programs leading to academic improvement include the following characteristics: making learning fun, interactive, and hands-on, delivering academic content that complements curricular standards, hiring experienced and trained teachers, keeping class sizes small, and encouraging parents to teach children how to become better readers. For disadvantaged students, making programs affordable and accessible, involving parents, and involving the community appear to be aligned with best practices.”

Method

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

(“summer school” OR “summer program”) AND (“best practices” OR “promising practices”)

Databases and Resources

We searched ERIC for relevant resources. ERIC is a free online library of over 1.6 million citations of education research sponsored by the Institute of Education Sciences. Additionally, we searched Google Scholar and PsychInfo.

Reference Search and Selection Criteria

When searching and selecting resources to include, we consider the criteria listed below.

• Date of the Publication: References and resources published within the last 15 years, from 2003 to present, were included in the search and review.
• Search Priorities of Reference Sources: Search priority is given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations and academic databases. Priority is also given to sources that provide free access to the full article.
• Methodology: Priority is given to the most rigorous study designs, such as randomized controlled trials and quasi-experimental designs, and we may also include descriptive data analyses, survey results, mixed-methods studies, literature reviews, or meta-analyses. Other considerations include the target population and sample, including their relevance to the question, generalizability, and general quality. Priority is given to publications that are peer-reviewed journal articles or reports reviewed by IES and other federal or federally funded organizations. If there are many research reports available, we select those with the strongest methodology, or the most recent of similar reports. When there are fewer resources available, we may include a broader range of information.

This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the West Region (Arizona, California, Nevada, Utah), which is served by the Regional Educational Laboratory West at WestEd. This memorandum was prepared by REL West under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-17-C-0012, administered by WestEd. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.