Project Activities
The research team iteratively developed the AIM Coaching package over the first 2 years of the project. The following year, the final version of the intervention was tested in a randomized controlled trial to determine its promise for improving teachers' knowledge of evidence-based literacy practices and students' reading outcomes. Finally, the research team examined issues related to sustainability of the model.
Structured Abstract
Setting
The research took place in urban and near-urban middle schools in Maryland, DC, and Texas.
Sample
In the first year, an advisory board of 6 instructional leaders and 10 teachers provided input to inform intervention development. Additionally, 141 middle school teachers and 64 instructional leaders completed surveys on the typical support teachers receive. The following year, two instructional leaders from the advisory board and the teachers in their school field tested the intervention. The remaining four instructional leaders from the advisory board provided continued consultation. During the pilot study, participants included 16 instructional leaders, 47 middle school teachers (of English language arts, social studies, or science), and approximately 1,200 students in grades 6-7. In the final year, participants included one instructional leader and nine treatment teachers from one school that participated in the pilot study.
The project resulted in the development of the AIM Coaching package, a multistage process that starts with PD for instructional leaders and is followed by implementation of AIM Coaching that includes a set of (a) decision points, (b) tailoring variables, (c) decision rules, and (d) intervention options to help instructional leaders systematically support teachers' implementation of an evidence-based reading comprehension intervention called Promoting Adolescents' Comprehension of Text (PACT) Plus. To implement AIM Coaching, instructional leaders provide the Stage 1 intervention for all teachers in weeks 1-6, which includes PD and standardized protocol support on implementing PACT Plus. At weeks 7 and 8, leaders enter the Progress Monitoring Stage. They consider teacher skill (based on each teacher’s average fidelity score across the three PACT Plus practices) and teacher will (based on each teacher’s average collaborative effort score) to determine each teacher’s support level: (1) High skill (regardless of will), (2) Low/average skill and high will, or (3) Low/average skill and low will. Instructional leaders then use a menu of intervention options to target teacher needs during Stage 2 (weeks 9-14). The Progress Monitoring Stage and Stage 2 repeat until the end of the school year.
Research design and methods
An iterative process was used to guide the development and testing of the AIM Coaching package. In the first year, the team used a mixed-methods approach to develop and refine the AIM Coaching package and fidelity forms. The team surveyed teachers and instructional leaders across the U.S. on the typical support teachers receive. Results were interpreted by researchers, an advisory board of instructional leaders, and teachers and used to inform the development of AIM Coaching and PD for instructional leaders. The advisory board also provided input on the initial versions of AIM Coaching and the PD. The following year, the team conducted a field trial of the AIM Coaching package with two instructional leaders from the advisory board and the teachers in their school to get feedback on the feasibility, usability, and perceived effectiveness of the AIM Coaching package. Based on the results of the field trial and teacher and instructional leader feedback, the team further revised the AIM Coaching package. Next, the team pilot tested the final AIM Coaching package in a small randomized controlled trial (RCT) to determine its promise for improving teacher and student outcomes. All teachers received training in PACT Plus, but intervention teachers received AIM Coaching support from instructional leaders while control teachers received typical coaching support. Finally, the team conducted a case study of one school that engaged in the pilot RCT, interviewing the school’s instructional leader and intervention teachers to learn about their perceptions of sustainability.
Control condition
In the pilot study, teachers in the control condition were provided with PACT Plus professional development from the research team and business-as-usual support from school-based instructional leaders.
Key measures
The research team used a variety of measures during the iterative development process, including two researcher-developed measures to learn about typical support provided to teachers and districts' and schools' context, focus groups and interviews, and questionnaires to obtain feedback from instructional leaders and teachers on the AIM Coaching package. During the pilot study, the team used measures that provided information on teacher perceptions of social validity and teacher self- and collective efficacy. Student outcomes were assessed using the Main Idea Measure and the Test of Sentence Reading Efficiency and Comprehension. A variety of measures were used to assess fidelity of the AIM Coaching package (instructional leaders' implementation of AIM Coaching and teachers' implementation of PACT Plus), including logs from instructional leaders, checklists, and audio recordings. A researcher-designed interview protocol was used to assess sustainability. Data on ingredients for the cost analysis were gathered through reviews of program documentation and budgets, interviews with project personnel, and surveys and interviews with school personnel.
Data analytic strategy
Qualitative data were coded for themes. Quantitative data from the iterative design process were analyzed descriptively and informed intervention development. Multilevel regression models were used to evaluate data from the pilot study and determine the promise of the intervention for improving teacher and student outcomes.
Cost analysis strategy
A cost analysis of the AIM Coaching package was conducted using the ingredients method. Personnel costs accounted for approximately 94% of total costs in both the AIM Coaching and business-as-usual coaching conditions. The total cost to implement AIM Coaching was $134,863 per school (or $5,619 per teacher; $280 per student), compared to $133,752 per school (or $4,053 per teacher; $118 per student) for the comparison condition. The results of the cost-effectiveness analysis indicate that AIM Coaching is somewhat more costly than the comparison group business-as-usual coaching, with small but generally positive effects on student reading outcomes. To some extent, the variation in engagement among teachers in the comparison group may explain the relatively higher cost for AIM Coaching. Given that this cost-effectiveness analysis is based on a pilot study of the AIM Coaching package, future large-scale efficacy trials of the package are likely to yield better estimates of its cost and effectiveness.
Key outcomes
The main findings of this project, as the principal investigator reports, are as follows:
- AIM Coaching was feasible to implement and improved teacher fidelity to PACT Plus implementation. AIM coaches achieved moderate-to-high adherence during the first complete implementation cycle (Stage 1, Progress Monitoring, and Stage 2), though fidelity declined in subsequent cycles suggesting coaches may require additional support to sustain coaching implementation across the school year. Teachers receiving AIM Coaching demonstrated large, statistically significant gains in adherence to PACT Plus instructional routines (ES = 1.50) and large, though not statistically significant, improvements in overall instructional quality and self- and collective efficacy.
- Students in both conditions performed comparably on reading measures. Students in the AIM Coaching and comparison groups showed similar gains on both the proximal main idea measure and standardized measure of general reading comprehension. These findings align with prior professional learning studies at the secondary level and are expected given the small sample size typical of pilot studies and the cascading nature of multi-level interventions where student outcomes depend on achieving and sustaining high teacher implementation fidelity over time.
- Key factors influencing sustainability of AIM Coaching practices include alignment with existing professional learning structures, perceived teacher buy-in, and the adaptability of AIM Coaching tools. Barriers to long-term sustainment included limited time for coaching, competing initiatives, and a perceived need for additional training to support teachers with lower readiness or motivation for engaging in coaching.
- Implications for Policy and Practice: The findings from this pilot study suggest that schools can feasibly integrate an adaptive intervention coaching model (AIM Coaching) into existing professional learning systems to strengthen middle school teachers’ implementation of evidence-based literacy practices (PACT Plus). Administrators play a critical role in the successful adoption and sustainment of AIM Coaching, including allocating protected time and staffing for coaching, supporting ongoing professional learning for coaches and teachers, setting clear expectations for staff participation, and aligning AIM Coaching within existing professional learning structures and initiatives.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Project website:
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Shelton, A., Swanson, E., Wexler, J., Payne, S. B., & Hogan, E. (2023). An exploration of middle school literacy coaching: A study of teachers and instructional coaches. Teacher Education and Special Education, 46(4), 300–316. https://doi.org/10.1177/08884064231172733
Wexler, J., Shelton, A., Swanson, E., Payne, S. B., Sayers, R., Johnston, T., Hogan, E. K., & Stark, K. (2025). The development of a coaching model: challenges and implications for intervention research. Remedial and Special Education, 7(2), 153-164 https://doi.org/10.1177/07419325241304126
Wexler, J., Swanson, E., & Shelton, A. (2021). Literacy coaching in secondary settings: Improving intensive instruction for all students. Guilford Press.
Payne S. B., Swanson, E., McGraw, R. L., Reutebuch, C. K., Smith Levitan, J. A., Hogan, E., & Wexler, J. (in press). Coaching rural schoolteachers to implement schoolwide literacy practices to improve reading outcomes for students with disabilities. In B. Cooke & T. Landrum (Eds.), Advances in learning and behavioral disabilities. Emerald.
Shelton, A., & Wexler, J. (2022). The development of reading comprehension in adolescents with literacy difficulties. In T. C. Aceves, E. Talbott, T. W. Farmer, D. L. Lee, & K. L. McMaster (Eds.), Handbook of special education research, Volume I (1st ed., pp. 272–284). Routledge. https://doi.org/10.4324/9781003156857-23
Wexler, J., Reed, D., Omohundro, K. M., (2025). What works to support struggling readers. In S. Neuman & M. Kuhn (Eds.) Handbook on the science of literacy in grades 3-8 (pp. 223-240). Guilford.
Wexler, J., Shelton, A., Stark, K., Hogan, E., Chow, J., & Fisk, R. (2024). Professional development as a pathway for sustaining teachers. In P. T. Sindelar, E. D. McCray, J. McLeskey, E. Bettini, & M. T. Brownell (Eds.), Handbook of research on special education teacher preparation (2nd ed., Vol. 1, pp. 319–340). Routledge. https://doi.org/10.4324/9781003297093-20
Payne, S. B., & Swanson, E. (2023). Supporting families to motivate their middle school student during homework time. TEACHING Exceptional Children, 55(6), 422-430.
Shelton, A., Hogan, E., Chow, J. C., & Wexler, J. (2023). A synthesis of professional development targeting literacy instruction and intervention for English learners. Review of Educational Research, 93(1), 37-72. https://doi.org/10.3102/00346543221087718
Shelton, A., & Wexler, J. (2024). Main idea strategy instruction to support middle school students with intellectual disability. TEACHING Exceptional Children, 56(5), 396-403 https://doi.org/10.1177/00400599221081036
Stark, K., Hogan, E., & Payne, B. (2024). Opening the “black box” of teacher development through coaching: A synthesis of coaching discourse for in-service teachers. Teacher Education Quarterly, 51(4), 67–96.
Stark, K., & Koslouski, J. (2022). The emotions of professional learning: Considerations for instructional leaders. NASSP Bulletin, 106(4), 281–297. https://doi.org/10.1177/01926365221134589
Stark, K., Sayers, R., & Angell, E. (2024). Documenting teachers’ motivation for professional learning: a systematic review. Professional Development in Education, 1–23. https://doi.org/10.1080/19415257.2024.2374332
Stark, K., Wexler, J., Shelton, A., Johnston, T. B., & Omohundro, K. (2024). Explicit and evidence-based literacy instruction in middle school: An observation study. Reading & Writing, 37(9), 2275–2276. https://doi.org/10.1007/s11145-023-10484-6
Additional project information
Additional online resources and information:
Online Modules: www.aimcoaching.org
Other Online Resources:
Sayers, R., Fisk, R., Shelton, A., Reutebuch, C., Payne, B. (2021). Helping your child become a self-regulated reader. AIM Coaching. https://aimcoaching.org/wp-content/uploads/2021/12/Parent_Flyer_SRR.pdf
Sayers, R., Fisk, R., Shelton, A., Reutebuch, C., Payne, B. (2021). Helping your students become self-regulated readers. AIM Coaching. https://aimcoaching.org/wp-content/uploads/2021/12/Teacher_Flyer_SRR.pdf
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
