Summer, Afterschool and Other Out-of-School Time Settings

Supporting Academic Learning and Development through Evidence-Based Practices A Practitioner’s Guide to Improving Literacy Outcomes for Students Chronic Absenteeism Summer, Afterschool and Other Out-of-School Time Settings Data Science Education General Resources

Purpose

States, districts and their partners have the opportunity to use out-of-school time (OST) programs to address the academic learning and developmental needs of learners of all ages. A wide range of programs can be considered OST programs, including comprehensive afterschool, summer-learning and enrichment programs, vacation academies, work-based learning programs, youth development programs, experiential or service-learning programs and other expanded learning programs.

IES is committed to supporting the use of evidence-based practices in the design and implementation of these programs–while acknowledging that, as a field, we continue to learn together what works best, for whom, and under what conditions.

This site catalogs a variety of resources, including those created outside of IES, that may be useful in this effort. Keep in mind that while some practices are based on rigorous evidence, many are rooted in theory but have yet to be tested. As we use and share today's best available evidence on out-of-school learning, we must continue to build stronger, more rigorous evidence about "what works" going forward.

Other Resources

Summer Learning & Enrichment Collaborative
Visit Site Read More Summer Learning & Enrichment Collaborative
National Center on Afterschool and Summer Enrichment
Visit Site Read More National Center on Afterschool and Summer Enrichment
National Mentoring Resource Center
Visit Site Read More National Mentoring Resource Center

Using Evidence in Out-of-School Time Programs

This site is organized around the idea of using evidence to inform the design, implementation, and continuous improvement of out-of-school time programs. While there are many models of evidence-based implementation and improvement, they generally follow similar steps. We use the cycle of evidence use and evidence building outlined in the Department's "Strengthening the Effectiveness of ESEA Investments."

Continuous improvement wheel

Begin by identifying the specific needs of your students and community and building awareness of the resources available to design and implement your out-of-school time program. We recommend taking the steps below, including:

  • Consult existing data and needs assessments to help define your program's purpose
    • Review local formative or summative assessment data to identify areas where students are performing below grade level
    • Consult the results of stakeholder consultation activities already conducted by your SEA or LEA
    • Consult out-of-school time program experts familiar with your state, such as your 21st Century Community Learning Centers State Education Contact
    • Explore resources from other sources that may add to your understanding of your state's context, such as information from:
  • Meaningfully engage stakeholders
    • Identify community stakeholders and consider how the stakeholders are aligned with the program or system goals. Consider:
      • Students, families, and caregivers
      • Teachers, other educators, school staff, principals, and other school leaders
      • Child advocacy organizations, including those representing the interests of children with disabilities, English learners, children experiencing homelessness, children in foster care, migratory students, and children are involved with the juvenile justice system
      • School and district administrators, including superintendents
      • Charter school leaders, if applicable
      • Education organizations and advocacy groups, including teacher and staff unions
      • Community and elected leaders, including tribal leaders, school boards, and leaders representing business and industry
      • Other potential partners, including other community-based organizations, university faculty and staff, and faith communities
  • Prioritize relevant outcomes to focus your program's activities
    • List the outcomes identified through needs sensing and data review
    • Consider other potential outcomes that have been explored in research related to out-of-school time programs.

Additional Resources from the U.S. Department of Education

  • The State Support Network's Needs Assessment Guidebook

Other Additional Resources (non-U.S. Department of Education)

Well-designed and well-implemented out-of-school time programs have the potential to support a variety of student and community outcomes. Whenever possible, you should design (or select) programs that use evidence-based practices for out-of-school time learning generally and that include evidence-based practices that are appropriate to the specific outcomes you identified in Step 1. We discuss both below. For further discussion of key terms, please visit AIR's Recognizing the Role of Research and Evidence in Out-of-School Time.

  • Be aware of evidence-based practices for use in out-of-school time learning
    • Review the recommendations from the What Works Clearinghouse's Structuring Out-of-School Time to Improve Academic Achievement Practice Guide below, which include:
    • Familiarize yourself with the following list of evidence-based practices for out-of-school time learning, as well as practices that have been recommended by out-of-school time learning experts but may not yet have rigorous evidence supporting their implementation:
      • Program infrastructure
        • Evidence-based practices include: plan early and well; use effective educators; and maintain a low child-to-adult ratio.
        • Recommended practices include: clearly defined program goals and program structures aligned to those goals; support for the physical, mental, and emotional well-being of all participants and a program mission and vision designed to motivate staff.
      • Program content and practices
        • Evidence-based practices include: offer sustained programming that students will attend regularly; plan for high-quality programming that includes both enrichment and academics; create a positive and welcoming site climate to encourage attendance (see attachment below).

        • Recommended practices include: include enrichment and academic content, make programs free, flexible and inclusive.

      • Program implementation and partnerships
  • Explore existing evidence-based programs and practices
  • Select programs that use evidence and build evidence of their effectiveness
    • As you consider which program or programs to select, vet programs to ensure:
      • They make the greatest possible use of the evidence-based practices identified above;
      • They have high-quality research that demonstrates the program has a positive effect on the desired student outcomes; and
      • The program has a plan to use data for improvement and evaluate their effectiveness.
    • Consider asking providers questions aligned to the evidence-based practices and expert recommendations above, including:
      • What are your program goals? How do daily activities align to program goals?
      • How does the program fit within the local context and does it address the needs of the community? Do you have evidence that supports this information?
      • How do you recruit, train, support and retain program staff? What are the required skills, credentials or competencies for staff?
      • How do you ensure physical and psychological safety of program participants and stuff? How do you create a welcoming and positive site climate?
      • How do you support the mental health and wellbeing of all program participants?
      • How do you foster strong relationships between staff and students? Staff and families?
      • How do you engage families and the community in planning and implementing your programs?
      • How do you ensure consistent attendance for students?
      • What is the length, duration and format of the programming?
      • What is your program outcome data? How do you continuously improve your program?
      • What is your evaluation plan for this program?

  • ED Research for Recovery's "Advancing Student Learning and Opportunity Through Voluntary Academic Summer Learning Programs"

  • AIR's Recognizing the Role of Research and Evidence in Out-of-School Time

  • The Afterschool Alliance's Key Principles for Expanding Learning to Support Student Re-Engagement

  • The Wallace Foundation's Evidence-based Considerations for COVID-19 Reopening and Recovery
    • Summer Learning with Academic and Non-academic Activities
    • The Importance of Adult Skills in SEL
  • Helpkidsrecover.org's compilation of Evidence-Based Strategies
  • Peterson, T.K. and Vandell, D.L. (2021) The Evidence Base for Summer Enrichment and Comprehensive Afterschool Opportunities. Washington, DC: Collaborative Communications
  • National Institute on Out-of-School Time Research and Resources

Planning for implementation requires dedicated time and capacity.

  • Build a logic model for your approach
    • Use your favorite logic model design or use REL Pacific's Logic Model tool or the template provided here (See Figure 2).
    • Ensure that your inputs include student attendance and that you have a way to track which students receive which programming and supports.
Logic Model Tool
  • Develop detailed implementation plans, timelines, and workplans
    • If you are planning an expanded learning program, start with the 21st Century Community Learning Center's New Director's Toolkit

    • If you are planning a summer program, consider using resources like the Summer Learning Toolkit which contains examples, templates and more to plan your program. You can also see how the State of Texas has adapted the toolkit to fit their needs in the attachment below.

    • If you haven't already, reach out to your 21st Century Community Learning Centers State Education Contacts or your 50 State Afterschool Network lead and ask them for planning tools or resources they have used previously.
  • Ensure sufficient time for staff recruitment and training
    • Build a plan or review the plan that your staff recruitment and training technical assistance provider give you.
    • Connect with your AmeriCorps State Service Commission to determine opportunities to utilize Corps members as staff members to scale rapidly
    • Secure a technical assistance contract for staff training and professional development. If not, use the Y4Y resources.
    • If feasible, recruit high school students as staff members to provide them with summer jobs and strong PD.
  • Ensure sufficient time for student recruitment and family engagement
    • Do you have partnerships with community-based organizations, family and community engagement experts or others?
  • Develop plans for continuous improvement, data collection, and evaluation

Other Additional Resources (non-U.S. Department of Education)

Implementing strong plans requires ensuring alignment with all of the key stakeholders and regularly checking in on plan implementation.

  • Implement your plans
    • Do you have regular touchpoints with all the relevant stakeholders?
    • Do you have ways to monitor progress?
    • Do you have plans in place for unforeseen challenges?
  • Collect data to inform future improvement efforts
    • Have you involved youth in survey design? It is a great way to get to know what matters most to them.
    • Do you have a way to track which students receive which services and how frequently?
    • Have you established your pre and post assessment tools?
    • Have you established your tools for daily, weekly, biweekly and monthly data collection?
      • In general, student attendance and engagement can be measured daily
      • Student mindsets and skills can be measured using formative and summative assessments.
      • Staff capacity and mindsets can be measured using formative and summative assessments.
  • Other Additional Resources (non-U.S. Department of Education)

As you implement your programming it is important to examine outcomes and data at regular intervals and refine your approach as applicable to continually improve, learn and share your results with key stakeholders.

  • Review data and refine your approach

    • Read and use the checklists and templates in the continuous improvement toolkit that IES and REL Northeast and Island published for schools and districts to use continuous improvement in education.

    • At minimum, write out your plan for reviewing data addressing:
      • How frequently?
      • With whom?
      • What questions are we trying to answer?
      • What will we do with what we learn?
  • Begin continuous improvement cycle again as appropriate
    • Using your logic model and continuous improvement plan, refine your approach and begin the continuous improvement cycle again starting with Step 3 – Plan for Implementation.
  • Additional Resources from the U.S. Department of Education