Evaluation of MyTeachingPartner-Secondary Delivered Using Local Coaches during the COVID-19 Pandemic: Evidence from a Randomized Experiment
Andrew J. Wayne; Mengli Song; Alex Bishop; Cheryl Graczewski; Sami Kitmitto; Heleana Lally (2023). American Institutes for Research. Retrieved from: https://eric.ed.gov/?id=ED630949
-
examining4,876Students, grades6-12
Single Study Review
Review Details
Reviewed: February 2025
- Single Study Review (findings for MyTeachingPartner-Secondary)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a cluster randomized controlled trial with low cluster-level attrition and individual-level non-response.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
||
|---|---|---|---|---|---|---|---|---|---|
|
Overall score (end-of-grade + end-of-course) |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Year 1;
|
-0.16 |
-0.18 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
ELA Score (end-of-grade + end-of-course) |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Year 1;
|
-0.18 |
-0.22 |
No |
-- | ||
|
ELA end-of-grade-score |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Year 1;
|
-0.11 |
-0.14 |
No |
-- | ||
|
Overall end-of-grade score |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Year 1;
|
-0.15 |
-0.15 |
No |
-- | ||
|
Math end-of-grade score |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Year 1;
|
-0.17 |
-0.16 |
No |
-- | ||
|
Math Score (end-of-grade + end-of-course) |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Year 1;
|
-0.14 |
-0.07 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
||
|---|---|---|---|---|---|---|---|---|---|
| Show Supplemental Findings | |||||||||
|
Teachers' sense of self-efficacy |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Full sample;
|
5.55 |
5.37 |
No |
-- | ||
|
Enthusiasm about teaching |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Full sample;
|
2.77 |
2.74 |
No |
-- | ||
|
Symptoms of depression |
MyTeachingPartner-Secondary vs. Business as usual |
-1 Years |
Full sample;
|
0.93 |
1.03 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Other or unknown: 100% -
Rural, Suburban, Urban
-
Race Other or unknown 100% -
Ethnicity Other or unknown 100% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
This study was conducted in 16 public middle (13) and high schools (3) in six districts located in five states. Schools were recruited based on having a large economically disadvantaged and minority student population. Teachers from study schools were recruited if they were not participating in an induction program involving regular mentoring. They also needed to teach a year-long mathematics or English language arts class that had an end-of-course exam or included mostly students at grade levels based on district testing.
Study sample
A total of 87 teachers (44 intervention, 43 comparison) were randomized to study condition within blocks defined by subjects and schools. The analytic sample included 42 intervention teachers and 40 comparison teachers. The number of students in the randomized sample was 4,895 (2,650 intervention, 2,245 comparison), with 4,876 (2,639 intervention, 2,237 comparison) in the analytic sample. About 41 percent of the teachers had 4-10 years of experience; 25 percent had 11-20 years of experience; and 24 percent had 21 or more years of teaching experience. About 57 percent had a graduate degree. The authors do not report demographic characteristics of the students.
Intervention Group
MyTeachingPartner-Secondary (MTP-S) is an instructional coaching program designed to improve teacher-student interactions and increase student engagement and achievement. The 2-year intervention includes two components: annual teacher trainings and ongoing coaching cycles. Teachers and local coaches complete 6-10 coaching cycles per year. Each coaching cycle included 5 steps: (1) the teacher video records their own classroom instruction; (2) the coach views the recording, selects three 1-minute clips of examples of classroom interactions and behaviors, writes prompts for the teacher to promote awareness of classroom interactions, the role the interactions played in student engagement and learning, and the role of the teacher in fostering these interactions; (3) the teacher views the video and prompts and posts responses in an online journal for the coach to read; (4) the teacher and coach meet for a 30-minute video- or in-person conference to discuss the edited video and the teacher’s responses to the prompts; and (5) the coach sends the teacher a summary of the conference and an action plan.
Comparison Group
The comparison teachers were told to deliver instruction as usual. They did not have access to any of the MyTeachingPartner materials or coaching.
Support for implementation
Teachstone provides significant support for implementation, including training, monitoring, and supporting local coaches. Teachstone also provides teachers a 20-hour orientation the first year and a refresher the second year.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, statistical significance, and sample size of the findings within a domain, the WWC assigns effectiveness ratings as one of the following: Tier 1 (strong evidence), Tier 2 (moderate evidence), Tier 3 (promising evidence), uncertain effects, and negative effects. For more detail, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).