Skip Navigation

REL Central Ask A REL Response

Beating the Odds

March 2021

Question

What educational activities or practices have been found to be effective in increasing students’ math, reading, or social and emotional knowledge or skills during summer school or extended school years?

Response

Following an established research protocol, REL Central conducted a search for research reports as well as descriptive study articles to help answer the question. The resources included ERIC and other federally funded databases and organizations, research institutions, academic databases, and general Internet search engines. (For details, please see the methods section at the end of this memo.)

References are listed in alphabetical order, not necessarily in order of relevance. We have not evaluated the quality of the references provided in this response, and we offer them only for your information. We compiled the references from the most commonly used resources of research, but they are not comprehensive and other relevant sources may exist.

Research References

Kidron, Y., & Lindsay, J. (2014). The effects of increased learning time on student academic and nonacademic outcomes: Findings from a meta-analytic review (REL 2014–015). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Appalachia. Retrieved from https://eric.ed.gov/?id=ED545233

From the ERIC abstract:

“REL Appalachia conducted a systematic review of the research evidence on the effects of increased learning time. After screening more than 7,000 studies, REL Appalachia identified 30 that met the most rigorous standards for research. A review of those 30 studies found that increased learning time does not always produce positive results. However, some forms of instruction tailored to the needs of specific types of students were found to improve their circumstances. Specific findings include: (1) Increased learning time promoted student achievement in mathematics and literacy when instruction was led by a certified teacher and when teachers used a traditional instructional style (i.e., the teacher is responsible for the progression of activities and students follow directions to complete tasks); (2) Increased learning time improved literacy outcomes for students performing below standards; and (3) Increased learning time improved social-emotional skills of students with attention deficit/hyperactivity disorder.”


Kidron, Y., & Lindsay, J. (2014). Stated Briefly: What does the research say about increased learning time and student outcomes? (REL 2015–061). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Appalachia. Retrieved from https://eric.ed.gov/?id=ED547261

From the ERIC abstract:

“REL Appalachia conducted a systematic review of the research evidence on the effects of increased learning time. After screening more than 7,000 studies, REL Appalachia identified 30 that met the most rigorous standards for research. A review of those 30 studies found that increased learning time does not always produce positive results. However, some forms of instruction tailored to the needs of specific types of students were found to improve their circumstances. Findings suggest that the impacts of these programs depend on the settings, implementation features, and types of students targeted. This ‘Stated Briefly’ report is a companion piece that summarizes the results of another report entitled ‘The effects of increased learning time on students’ academic and nonacademic outcomes,’ released on July 9, 2014.”


Le Floch, K. C., Birman, B., O’Day, J., Hurlburt, S., Mercado-Garcia, D., Goff, R., Manship, K., Brown, S., Therriault, S. B., Rosenberg, L., Angus, M. H., & Hulsey, L. (2014). Case studies of schools receiving school improvement grants: Findings after the first year of implementation (NCEE 2014–4015). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from https://eric.ed.gov/?id=ED545112

From the ERIC abstract:

“The Study of School Turnaround examines the improvement process in a purposive sample of 35 case study schools receiving federal School Improvement Grants (SIG) over a three-year period (2010–11 to 2012–13 school years). Using site visit, teacher survey, and fiscal data, the case studies describe the school contexts, the principals’ leadership styles, the schools’ improvement strategies and actions, the supports states and districts provide to the schools, school stakeholders’ perceptions of improvement, and how SIG fits into the schools’ change process. Findings after the first year of implementation in the 25 ‘core’ sample schools reveal that while all were low-performing, the schools differed in their community and fiscal contexts, performance and reform histories, interpretations of the causes of–and potential solutions for–their performance problems, and perceptions of improvement after the first year of SIG. However, most schools did report that their improvement strategies and actions during the first year of SIG were a continuation of activities or plans that predated SIG, and few schools appeared to have experienced a disruption from past practice as of spring 2011.”


Mac Iver, M. A., & Mac Iver, D. J. (2015). The Baltimore City Schools Middle School STEM Summer Program with VEX Robotics. Baltimore Education Research Consortium. Retrieved from https://eric.ed.gov/?id=ED570654

From the ERIC abstract:

“In 2011 Baltimore City Schools submitted a successful proposal for an Investing in Innovations (i3) grant to offer a three year (2012–2014) summer program designed to expose rising sixth through eighth grade students to VEX robotics. The i3-funded Middle School Science, Technology, Engineering and Mathematics (STEM) Summer Learning Program was part of a larger Baltimore City STEM summer learning program entitled ‘Create the Solution’ in 2012 and ‘22nd Century Pioneers’ in 2013 and 2014. The five-week summer program offered in 2012, 2013, and 2014 consisted of a half-day of instruction in mathematics and science and a half-day of enrichment activities. The robotics workshop taught students the fundamentals of building robots and provided time for teams to build their own robots and participate in competitions. The larger program offered different enrichment activities such as sports or arts. This report addresses research questions regarding the program’s: (1) implementation fidelity; (2) performance goals; (3) impact on student attendance and mathematics achievement outcomes; (4) impact on student aspirations for college, studying STEM subjects in college, and pursuing STEM careers; and (5) impact on measures of teacher effectiveness. The following includes a summary for each: (1) Implementation Fidelity: Instruction in mathematics and robotics was implemented with fidelity all three program years. Implementation fidelity was lower for the professional development in robotics and mathematics components of the program because teacher attendance rates did not meet the thresholds set by City Schools; (2) Enrollment Goals: Most program enrollment goals were not met. Enrollment in the i3-funded program was 193 students in 2012 (goal 400), 384 in 2013 (goal 500), and 386 in 2014 (goal 600). The program sought to enroll 80% low-performing students in mathematics each year, but fell significantly short of this goal despite the district’s efforts to reach out to these students. In addition, the program goal of enrolling at least 50% female participants was not met. The program also sought to have at least 80% of students attend at least 70% of the time (17 of the 24 program days), but only 55% of students attended at that rate. The program did meet its goals for recruiting minority (at least 95%) and high poverty students (at least 80%) each year; (3) Program Impacts on Attendance: Found a significant program effect on attendance in the year following the 2012 program. Program students had average attendance rates of 1.4 percentage points higher than the comparison group the year following the program (97.0% vs. 95.6%). An even larger significant program effect for low-achieving students’ attendance was found in the year following the 2012 program (96.4% vs. 93.8%). The 2013 program students had slightly but not significantly higher attendance rates than their matched comparison students in the year following the program. The authors also ... examined whether there was still a program effect on attendance a year later (2013–14) for the Summer 2012 participants. Program participants had average attendance rates of 1.5 percentage points higher than comparison students (95.2% vs. 93.7%). Among the low-achieving students the attendance difference was 2.4 percentage points (93.6% for program students vs. 91.2% for comparison students). These effects were not statistically significant; (4) Program Impacts on Mathematics Achievement: There were no program effects on mathematics achievement for either the 2012 or 2013 programs; (5) Program Impacts on Student Aspirations: There was no evidence from student survey data that the robotics program had a positive effect on student aspirations to attend college, study STEM subjects in college, or pursue a STEM career for either the 2013 or 2014 programs; and (6) Program Impacts on Teacher Effectiveness: Analyses based on mean instructional effectiveness scores from Spring 2013 and Fall 2013 on the nine components of the district’s teacher evaluation tool examined whether teachers who received the summer professional development in 2013 made gains in instructional effectiveness. The difference between program teachers’ effectiveness scores before and after the professional development was not statistically significant. Data were not available to examine differences between program teachers and a comparable group of teachers who did not receive the summer professional development.”


Redford, J., Burns, S., & Hall, L. J. (2018). The summer after kindergarten: Children’s experiences by socioeconomic characteristics (NCES 2018–160). U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. Retrieved from https://eric.ed.gov/?id=ED583184

From the ERIC abstract:

“This Statistics in Brief investigates some factors that may contribute to achievement gaps in the summer after kindergarten, including differences in children’s experiences, such as participation in summer care arrangements, programs, and activities. Specifically, the brief describes students’ summer nonparental care arrangements, program attendance (e.g., at summer camps or summer school), participation in activities with family members in a typical week, and places visited with family members. Participation in summer activities is compared by two socioeconomic characteristics– household poverty status and parents’ highest level of education–utilizing data from the Early Childhood Longitudinal Study, Kindergarten Class of 2010–11.”


Williams, R., Rudo, Z., & Austin, M. (2020). District changes in student achievement and local practice under Georgia’s district and school flexibility policy (REL 2021–051). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from https://eric.ed.gov/?id=ED609818

From the ERIC abstract:

“In 2007 Georgia instituted a flexibility policy through which school districts enter into performance contracts with the state, receiving waivers from state rules, provisions, and guidelines in exchange for agreeing to meet annual accountability targets. The performance contracts are intended to incentivize innovations that increase achievement among all students. Between 2008/09 and 2016/17, 178 of Georgia’s 180 districts entered into a performance contract. The Georgia Department of Education requested an analysis of how student achievement changed after districts adopted a performance contract and what factors were related to those changes. The department also requested information on how districts used their performance contract to prioritize innovations in local practice. Overall, the study found little evidence that changes in student achievement coincided with adopting a performance contract but found significant variation in changes in achievement across districts, once other factors were adjusted for. Changes in achievement were largely unrelated to district characteristics, including urbanicity, timing of performance contract adoption, and district type, or features of the performance contract. District leaders reported prioritizing innovations related to college and career readiness, teacher certification requirements, instructional spending, and funding for school improvement after adopting a performance contract. Leaders perceived broad benefits from the priority innovations they identified, especially for staff and school climate, but also indicated that waivers were not required to implement many of the innovations. Despite the perceived benefits, changes in achievement were largely unrelated to the academic, human resources, and financial innovations that district leaders reported prioritizing after adopting a performance contract.”



Additional Organizations to Consult

Summer Matters: http://www.summermatters.net/

From the website:

“The vision of Summer Matters is that all young people in California have equitable access to high quality summer learning opportunities to support their year-round learning and well-being.”



Methods

Keywords and Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • “Extended School Year”
  • “Extended School Year” + activities
  • “School flexibility”
  • “Summer Programs”
  • “Summer School”
  • “Summer School” + activities
  • “Summer School” + literacy
  • “Summer School” + math
  • “Summer School” + reading
  • “Summer School” + “social emotional learning”

Databases and Resources

REL Central searched ERIC for relevant references. ERIC is a free online library, sponsored by the Institute of Education Sciences, of over 1.6 million citations of education research. Additionally, we searched Google Scholar and Google.

Reference Search and Selection Criteria

When searching for and reviewing references, REL Central considered the following criteria:

  • Date of the Publication: The search and review included references published between 2011 and 2021.
  • Search Priorities of Reference Sources: Search priority was given to ERIC, followed by Google Scholar and Google.
  • Methodology: The following methodological priorities/considerations were used in the review and selection of the references: (a) study types, such as randomized controlled trials, quasi-experiments, surveys, descriptive analyses, and literature reviews; and (b) target population and sample.

This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Central Region (Colorado, Kansas, Missouri, Nebraska, North Dakota, South Dakota, Wyoming), which is served by the Regional Educational Laboratory Central at Marzano Research. This memorandum was prepared by REL Central under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-17-C-0005, administered by Marzano Research. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.