Education Agency Partners: School districts in Kansas City and St. Louis, Missouri and in Austin and Dallas, Texas

Co-Principal Investigators: Ketterlin Geller, Leanne; Powell, Sarah; Wiedermann, Wolfgang

Purpose: The purpose of this project is to evaluate different levels of intensity of a professional development (PD) and coaching model for middle school special education teachers to accelerate pandemic recovery in mathematics for their students with disabilities. Due to pandemic-related school closures, many students have not received enough instruction to develop a deep understanding of mathematics content. Students who experienced mathematics difficulties prior to the pandemic are at even greater risk for mathematics failure and students who did not experience difficulties pre-pandemic may be more likely to demonstrate challenges now. To address this, the research team will partner with several school districts in Missouri and Texas to implement and evaluate an adaptive PD and coaching model, STAIR (Supporting Teaching of Algebra: Individual Readiness), for special education teachers. STAIR is designed to increase teachers' use of data-based decision-making, evidence-based mathematics instruction, and formative assessment to inform their instructional adaptations and subsequently improve students' readiness for algebra. A sequential multiple assignment randomized trial (SMART) will be used to determine the effects of STAIR intensity on outcomes for teachers and students with disabilities.

Project Activities: The research team will utilize a SMART design to test the impact of different levels of intensity of STAIR, which will include light versus intense coaching in the first year, followed by additional peer mentoring or booster sessions for those teachers who did not respond to initial PD/coaching. The team will also assess the sustainability of impacts and examine the costs and cost-effectiveness of the different levels of STAIR intensity.

Products: This project will provide evidence for the impact of different levels of STAIR intensity on teachers' knowledge and practice and the mathematics outcomes of students with disabilities. The project will also result in peer-reviewed publications and presentations as well as products for education stakeholders, including mini-briefs with infographics and downloadable intervention materials. Results will be shared through social media and with the IES Regional Education Labs and the Education Service Centers across Texas.

Structured Abstract

Setting: The research will take place in 24 middle schools in suburban, urban, and rural areas in Missouri and Texas.

Sample: Participants will include 72 middle school special education teachers who provide mathematics instruction to students with disabilities (3 per school) and 432 students with disabilities (6 students per teacher). Students will be eligible for participation if they have an intensive learning need in mathematics, as evidenced by an Individualized Education Program goal in mathematics and are preparing to take algebra in high school.

Intervention: STAIR is a PD and coaching framework that provides training for teachers on strategies they can integrate into their mathematics instruction within their current curriculum. All teachers (regardless of the level of coaching assigned) participate in core and tailored PD. Core PD covers data-based decision making, assessment (screening, progress monitoring, and graphing of student progress), and evidence-based mathematics strategies (explicit and systematic instruction, use of precise mathematics language, use of multiple representations, fluency building, and word-problem instruction). Tailored PD occurs as part of coaching and uses videos to support targeted teacher learning. To address the needs of the educational partners for a cost-efficient way of improving special education teachers' practices, various levels of coaching intensity will be tested. The first-stage intervention will include either light or intense coaching. Teachers who receive intense coaching will meet one-on-one with a coach on a weekly basis, using a clear coaching routine. Teachers who receive light coaching will meet with coaches every other week as part of their typical data team or professional learning community meeting with a small group of teachers. In the second-stage intervention, teachers who are not responding to the first-stage intervention will continue to receive the originally assigned level of coaching and be randomly assigned to receive one of two enhanced treatments: coaching (either light or intense) + peer mentoring with a district liaison or coaching (either light or intense) + booster sessions targeted to address areas of need. Teachers assigned to peer mentoring will meet with a district liaison every other week to discuss STAIR related topics. Teachers assigned to booster sessions will receive support to meet their particular needs.

Research Design and Methods: The research team will use a SMART design to evaluate the efficacy of different levels of STAIR intensity for improving teacher and student outcomes. In Year 1, at the first stage, all teachers will be randomly assigned to either intense or light coaching to examine whether a less intense coaching routine that more closely matches typical school procedures is as effective as intense coaching. In Year 2, teachers who are not responding will be identified based on their scores on a researcher-developed measure of teachers' knowledge, instruction, and assessment practices. Teachers who are nonresponsive will be re-randomized to receive the second-stage intervention, which includes either augmenting the first-stage intervention with peer mentoring or booster sessions to address areas of need. Responders to the first-stage intervention will continue receiving the originally assigned intervention. In Year 3, the research team will assess the sustainability of STAIR impacts. Teachers will be administered measures at pre-, post-test, and follow-up to determine the initial effect and sustainability of STAIR intensity. The team will also collect observational data to assess fidelity and instructional quality. Teachers will also be interviewed (or participate in a focus group) at the end of each year to better understand the strengths and challenges of implementing data-based individualization with their students. At the student level, outcome data will be collected at pre- and post-test and progress monitoring data will be collected throughout. Researchers will conduct cost and cost-effectiveness analyses of the STAIR interventions.

Control Condition: Due to the nature of the research design, comparisons are made among treatment conditions, but there is no control condition.

Key Measures: The following measures will be used to assess teacher outcomes: the Integrated Knowledge and Motivation Assessment (teachers' knowledge and disposition on effective practices for teaching multiplicative reasoning); the Mathematics Teaching Efficacy Belief Instrument (teacher self-efficacy); and a researcher-developed measure of teachers' knowledge, instruction, and assessment practices related to data-based individualization. Teachers' fidelity of implementing evidence-based practices will be assessed monthly using the Recognizing Effective Special Education Teachers observational rubrics. Teachers will also be interviewed at post-test and follow up to assess their perceptions of implementing data-based individualization with their students. At pre-test, students will be administered the following measures to characterize the sample and contextual potential findings: Diagnostic Online Mathematics Assessment – Pre-Algebra (algebra readiness), the Woodcock-Johnson-IV (reading comprehension), and the Wide Range Achievement Test, Fifth Edition (word reading). Student outcomes will be assessed using standardized tests (The Stanford Achievement Test-Tenth Edition, Iowa Algebra Aptitude Test), state tests (the Texas STAAR Mathematics test for students in Texas and the Missouri Assessment Program test for those in Missouri), and a measure of mathematics anxiety (Mathematics Anxiety Rating Scale). Aimsweb will also be used to monitor students' progress on mathematics concepts and computation.

Data Analytic Strategy: The research team will evaluate 1) differences between first-stage intervention groups, 2) differences between second-stage intervention groups, and 3) differences between the four embedded adaptive intervention groups—intense coaching, light coaching, peer coaching, and individual booster sessions. To compare embedded interventions, the research team will use weighted-and-replicated regression. Analyses of covariance will be used to evaluate differences in teacher outcomes between first- and second-stage intervention groups. To account for the nesting of students within teachers, multilevel modeling will be used to evaluate effects on student outcomes. Interviews and focus groups will be transcribed and analyzed using a priori codes and emergent coding to identify major themes.

Cost Analysis: A cost analysis for each first-stage intervention (light and intense coaching) and second-stage intervention (boosters and peer mentoring) will be conducted following the ingredients method. Costs will be estimated from the perspective of the teacher. The researchers will derive both the total and incremental, average unit-cost to deliver STAIR based upon the estimated cost of implementation divided by the number of students served. Researchers will also evaluate an incremental cost-effectiveness ratios of the interventions.