Setting
The study was conducted in 12 middle and high schools in Virginia.
Study sample
The study recruited secondary school teachers who expected to be the primary instructor of
a course that had end-of-course standardized achievement exams. Participating teachers in
each school were grouped by the subject (i.e., math/science, language arts/social studies)
they would teach in the evaluation. Within each subject in each school, participating teachers
were randomly assigned to either the MTP-S intervention or “business-as-usual” professional
development. Teachers were then asked to select their “focal class” for inclusion in the
evaluation—the class that they expected to be the most academically challenging and that
had end-of-year achievement tests. Although initial selection of focal classes occurred before
teachers learned their research condition, some teachers later changed their focal classes.
Written parental consent and student assent were obtained for participating students in the
focal classes after randomization. Because the teachers’ ultimate selection of focal classes,
parental consent, and student assent could have been affected by knowledge of the teacher’s
research condition, the study was determined by the WWC to have a compromised random
assignment process.
The study included 78 teachers (40 MTP-S and 38 comparison) from 12 schools. The analytic
sample for the intervention year included 1,267 students (606 MTP-S and 661 comparison)
taught by 76 teachers (39 MTP-S and 37 comparison). The analytic sample for the post-intervention
year was composed of a new cohort of 970 students (419 MTP-S and 551 comparison)
who had not participated in the study during the previous year. These post-intervention
year students were taught by 61 teachers (27 MTP-S and 34 comparison) who had participated
in the study during the intervention year; however, the MTP-S teachers did not receive
MTP-S coaching during the post-intervention year.
The mean grade of the students across both cohorts was grade 8. Forty-six percent of students
were female, and 29% had families with incomes at less than 200% of the poverty line.
The racial/ethnic demographics were as follows: 72% were White, 22% were Black, 4% were
Hispanic, 2% were Asian, and 1% were of other race/ethnicity.
Intervention Group
Intervention group teachers attended an initial workshop, where consultants defined the MTP-S
principles and described the dimensions of high-quality teacher–student interactions from the
CLASS-S. Approximately twice a month throughout the school year, each teacher submitted
videotaped sessions from his or her focal class to a consultant, who identified brief segments
of the session for review and discussion. The teacher reviewed the identified segments for his
or her own behaviors and for student reactions and answered questions from the consultant
about the connection between the behaviors and the reactions. The teacher then conferred
with the consultant in a 20- to 30-minute discussion by phone in which the consultant recommended
strategies to enhance teacher–student interactions. Consultants also directed teachers
to view annotated video exemplars of high-quality teaching available on the MTP-S website.
The year of coaching was followed by a brief “booster” workshop. The intervention involved a
total of about 20 hours of in-service training over 13 months. The mean years of teaching experience
was 7.6 during the intervention year. Sixty-nine percent of MTP-S teachers were female,
and 59% held a master’s or higher level degree during the intervention year.
Comparison Group
Comparison group teachers received regular in-service training. They videotaped six classroom
segments throughout the school year at times that coincided with MTP-S teacher
videotapings, but they did not receive feedback on the videotapes. Comparison teachers did
not have access to the MTP-S library of video exemplars. The mean years of teaching experience
was 10.1 during the intervention year. Fifty-seven percent of comparison teachers were
female, and 70% held a master’s or higher level degree during the intervention year.
Outcome descriptions
An outcome in the general achievement domain was reported. Student achievement was measured
using end-of-year scores from tests taken in core subjects, with the score on the most
comparable course in the previous school year used as a pretest. Because the authors report
a single test score without distinguishing academic achievement in specific subject areas,
the outcome falls in the general achievement domain. For a more detailed description of this
outcome measure, see Appendix B.
Support for implementation
Prior to the beginning of the school year, both MTP-S and comparison group teachers participated
in a 3-hour workshop in which they received instruction in procedures for videotaping
classes and submitting the videotapes (as well as procedures for study-related tasks, such as
obtaining student assent/parent consent and collecting self-reported data). Consultants for the
MTP-S teachers were master teachers trained in using the CLASS-S system. Two consultants
led an initial workshop for intervention teachers that outlined MTP-S principles and discussed
the CLASS-S dimensions of high-quality teacher–student interactions.