Purpose
States, districts and their partners have the opportunity to use out-of-school time (OST) programs to address the academic learning and developmental needs of learners of all ages. A wide range of programs can be considered OST programs, including comprehensive afterschool, summer-learning and enrichment programs, vacation academies, work-based learning programs, youth development programs, experiential or service-learning programs and other expanded learning programs.
IES is committed to supporting the use of evidence-based practices in the design and implementation of these programs–while acknowledging that, as a field, we continue to learn together what works best, for whom, and under what conditions.
This site catalogs a variety of resources, including those created outside of IES, that may be useful in this effort. Keep in mind that while some practices are based on rigorous evidence, many are rooted in theory but have yet to be tested. As we use and share today's best available evidence on out-of-school learning, we must continue to build stronger, more rigorous evidence about "what works" going forward.
Other Resources
Using Evidence in Out-of-School Time Programs
This site is organized around the idea of using evidence to inform the design, implementation, and continuous improvement of out-of-school time programs. While there are many models of evidence-based implementation and improvement, they generally follow similar steps. We use the cycle of evidence use and evidence building outlined in the Department's "Strengthening the Effectiveness of ESEA Investments."

Begin by identifying the specific needs of your students and community and building awareness of the resources available to design and implement your out-of-school time program. We recommend taking the steps below, including:
- Consult existing data and needs assessments to help define your program's purpose
- Review local formative or summative assessment data to identify areas where students are performing below grade level
- Consult the results of stakeholder consultation activities already conducted by your SEA or LEA
- Consult out-of-school time program experts familiar with your state, such as your 21st Century Community Learning Centers State Education Contact
- Explore resources from other sources that may add to your understanding of your state's context, such as information from:
- The National Center on Safe Supportive Learning Environments
- Your 50 State Afterschool Network Lead (non-Department of Education)
- America After 3pm (non-Department of Education)
- Meaningfully engage stakeholders
- Identify community stakeholders and consider how the stakeholders are aligned with the program or system goals. Consider:
- Students, families, and caregivers
- Teachers, other educators, school staff, principals, and other school leaders
- Child advocacy organizations, including those representing the interests of children with disabilities, English learners, children experiencing homelessness, children in foster care, migratory students, and children are involved with the juvenile justice system
- School and district administrators, including superintendents
- Charter school leaders, if applicable
- Education organizations and advocacy groups, including teacher and staff unions
- Community and elected leaders, including tribal leaders, school boards, and leaders representing business and industry
- Other potential partners, including other community-based organizations, university faculty and staff, and faith communities
- Identify community stakeholders and consider how the stakeholders are aligned with the program or system goals. Consider:
- Prioritize relevant outcomes to focus your program's activities
- List the outcomes identified through needs sensing and data review
- Consider other potential outcomes that have been explored in research related to out-of-school time programs.
Additional Resources from the U.S. Department of Education
- The State Support Network's Needs Assessment Guidebook
Other Additional Resources (non-U.S. Department of Education)
- STEM Next's Changing the Game in STEM with Family Engagement
- The Wallace Foundation's Afterschool Programs: A Review of Evidence Under the Every Student Succeeds Act
- Putting Data to Work for Young People: A Framework for Measurement, Continuous Improvement and Equitable Systems a resource from Every Hour Counts
Well-designed and well-implemented out-of-school time programs have the potential to support a variety of student and community outcomes. Whenever possible, you should design (or select) programs that use evidence-based practices for out-of-school time learning generally and that include evidence-based practices that are appropriate to the specific outcomes you identified in Step 1. We discuss both below. For further discussion of key terms, please visit AIR's Recognizing the Role of Research and Evidence in Out-of-School Time.
- Be aware of evidence-based practices for use in out-of-school time learning
- Review the recommendations from the What Works Clearinghouse's Structuring Out-of-School Time to Improve Academic Achievement Practice Guide below, which include:
- Aligning the program academically with the school day
- Maximizing student participation and attendance
- Adapting instruction to individual and small group needs
- Providing engaging learning experiences
Assessing program performance and the use of results to improve the quality of the program
- Familiarize yourself with the following list of evidence-based practices for out-of-school time learning, as well as practices that have been recommended by out-of-school time learning experts but may not yet have rigorous evidence supporting their implementation:
- Program infrastructure
- Evidence-based practices include: plan early and well; use effective educators; and maintain a low child-to-adult ratio.
- Recommended practices include: clearly defined program goals and program structures aligned to those goals; support for the physical, mental, and emotional well-being of all participants and a program mission and vision designed to motivate staff.
- Program content and practices
- Evidence-based practices include: offer sustained programming that students will attend regularly; plan for high-quality programming that includes both enrichment and academics; create a positive and welcoming site climate to encourage attendance (see attachment below).
Recommended practices include: include enrichment and academic content, make programs free, flexible and inclusive.
- Program implementation and partnerships
- Evidence-based practices include: build strong partnerships; assess outcomes aligned to the program's focus and use of data as part of continuous improvement loops (see attachment below).
Recommended practices include: engaging families and caregivers in planning and activities; establishing meaningful collaboration with community
Out-of-School Time Program Evaluation (870.12 KB)
- Program infrastructure
- Review the recommendations from the What Works Clearinghouse's Structuring Out-of-School Time to Improve Academic Achievement Practice Guide below, which include:
- Explore existing evidence-based programs and practices
- Look in the What Works Clearinghouse (WWC) for a list of programs that have evidence aligned to the outcomes you are designing for. Helpful categories in the WWC include Literacy, Math, Science, Early Childhood, Behavior and High School graduation. (Not all these programs have been designed for use in out- of- school time programs, but they could be strong places to start.)
- Consult other non-Department of Education resources that reviewed the evidence for specific out-of-school time programs, including:
- Select programs that use evidence and build evidence of their effectiveness
- As you consider which program or programs to select, vet programs to ensure:
- They make the greatest possible use of the evidence-based practices identified above;
- They have high-quality research that demonstrates the program has a positive effect on the desired student outcomes; and
- The program has a plan to use data for improvement and evaluate their effectiveness.
- Consider asking providers questions aligned to the evidence-based practices and expert recommendations above, including:
- What are your program goals? How do daily activities align to program goals?
- How does the program fit within the local context and does it address the needs of the community? Do you have evidence that supports this information?
- How do you recruit, train, support and retain program staff? What are the required skills, credentials or competencies for staff?
- How do you ensure physical and psychological safety of program participants and stuff? How do you create a welcoming and positive site climate?
- How do you support the mental health and wellbeing of all program participants?
- How do you foster strong relationships between staff and students? Staff and families?
- How do you engage families and the community in planning and implementing your programs?
- How do you ensure consistent attendance for students?
- What is the length, duration and format of the programming?
- What is your program outcome data? How do you continuously improve your program?
- What is your evaluation plan for this program?
- As you consider which program or programs to select, vet programs to ensure:
ED Research for Recovery's "Advancing Student Learning and Opportunity Through Voluntary Academic Summer Learning Programs"
- AIR's Recognizing the Role of Research and Evidence in Out-of-School Time
The Afterschool Alliance's Key Principles for Expanding Learning to Support Student Re-Engagement
- The Wallace Foundation's Evidence-based Considerations for COVID-19 Reopening and Recovery
- Summer Learning with Academic and Non-academic Activities
- The Importance of Adult Skills in SEL
- Helpkidsrecover.org's compilation of Evidence-Based Strategies
- Peterson, T.K. and Vandell, D.L. (2021) The Evidence Base for Summer Enrichment and Comprehensive Afterschool Opportunities. Washington, DC: Collaborative Communications
- National Institute on Out-of-School Time Research and Resources
Planning for implementation requires dedicated time and capacity.
- Build a logic model for your approach
- Use your favorite logic model design or use REL Pacific's Logic Model tool or the template provided here (See Figure 2).
- Ensure that your inputs include student attendance and that you have a way to track which students receive which programming and supports.

- Develop detailed implementation plans, timelines, and workplans
- If you are planning an expanded learning program, start with the 21st Century Community Learning Center's New Director's Toolkit
If you are planning a summer program, consider using resources like the Summer Learning Toolkit which contains examples, templates and more to plan your program. You can also see how the State of Texas has adapted the toolkit to fit their needs in the attachment below.
- If you haven't already, reach out to your 21st Century Community Learning Centers State Education Contacts or your 50 State Afterschool Network lead and ask them for planning tools or resources they have used previously.
- Ensure sufficient time for staff recruitment and training
- Build a plan or review the plan that your staff recruitment and training technical assistance provider give you.
- Connect with your AmeriCorps State Service Commission to determine opportunities to utilize Corps members as staff members to scale rapidly
- Secure a technical assistance contract for staff training and professional development. If not, use the Y4Y resources.
- If feasible, recruit high school students as staff members to provide them with summer jobs and strong PD.
- Ensure sufficient time for student recruitment and family engagement
- Do you have partnerships with community-based organizations, family and community engagement experts or others?
- Develop plans for continuous improvement, data collection, and evaluation
Read and use the templates provided in the continuous improvement toolkit that IES and REL Northeast and Island published for schools and districts to use continuous improvement in education.
Establish a plan for continuous improvement and if applicable rigorous evaluation of your program. A great place to start is the "Putting Data to Work for Young People" by RAND Corporation, a step by step plan for collecting, analyzing and managing data to improve decision making or Harvard Family Research Project's Afterschool Evaluation 101 report.
Putting Data to Work for Young People (2.39 MB)- If you are running a STEM program, A Program Director's Guide to Evaluating STEM Education Programs, review an National Science Foundation (NSF) supported TA tool.
Other Additional Resources (non-U.S. Department of Education)
Implementing strong plans requires ensuring alignment with all of the key stakeholders and regularly checking in on plan implementation.
- Implement your plans
- Do you have regular touchpoints with all the relevant stakeholders?
- Do you have ways to monitor progress?
- Do you have plans in place for unforeseen challenges?
- Collect data to inform future improvement efforts
- Have you involved youth in survey design? It is a great way to get to know what matters most to them.
- Do you have a way to track which students receive which services and how frequently?
- Have you established your pre and post assessment tools?
- Have you established your tools for daily, weekly, biweekly and monthly data collection?
- In general, student attendance and engagement can be measured daily
- Student mindsets and skills can be measured using formative and summative assessments.
- Staff capacity and mindsets can be measured using formative and summative assessments.
- Other Additional Resources (non-U.S. Department of Education)
"Putting Data to Work for Young People" by RAND corporation is a step by step plan for collecting, analyzing and managing data to improve decision making.
Putting Data to Work for Young People (2.39 MB)- Are you ready to assess social and emotional learning? A Resource from AIR.
The Harvard Family Research Project compiled a resource to summarize a few different ways to collect data in Out of School Time Evaluations.
As you implement your programming it is important to examine outcomes and data at regular intervals and refine your approach as applicable to continually improve, learn and share your results with key stakeholders.
- Review data and refine your approach
Read and use the checklists and templates in the continuous improvement toolkit that IES and REL Northeast and Island published for schools and districts to use continuous improvement in education.
- At minimum, write out your plan for reviewing data addressing:
- How frequently?
- With whom?
- What questions are we trying to answer?
- What will we do with what we learn?
- Begin continuous improvement cycle again as appropriate
- Using your logic model and continuous improvement plan, refine your approach and begin the continuous improvement cycle again starting with Step 3 – Plan for Implementation.
- Additional Resources from the U.S. Department of Education
- REL Midwest compiled a resource roundup on Continuous Improvement Cycles that summarizes some of the key components of using your data to inform improvement.
- REL West pulled together a few resources to support educators and program staff to utilize learning huddles to shift instruction.