
An Interaction-Based Approach to Enhancing Secondary School Instruction and Student Achievement
Allen, Joseph; Pianta, Robert; Gregory, Anne; Mikami, Amori; Lun, Janetta (2011). Science v333 n6045 p1034-1037. Retrieved from: https://eric.ed.gov/?id=ED556046
-
examining1,267Students, grades6-12
My Teaching Partner-Secondary (MPT-S) Intervention Report - Teacher Training, Evaluation, and Compensation
Review Details
Reviewed: June 2015
- Randomized controlled trial
- Meets WWC standards with reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for My Teaching Partner-Secondary (MPT-S).
Findings
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
---|---|---|---|---|---|---|---|---|
Standards of Learning (SOL) |
My Teaching Partner-Secondary (MPT-S) vs. Business as usual |
Posttest |
Post-intervention year;
|
498.00 |
482.20 |
Yes |
|
|
Commonwealth of Virginia Standards of Learning (SOL) tests |
My Teaching Partner-Secondary (MPT-S) vs. Business as usual |
Posttest |
Intervention year;
|
468.90 |
465.60 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 46%
Male: 54% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Virginia
-
Race Asian 2% Black 22% Other or unknown 1% White 72% -
Ethnicity Hispanic 4% Not Hispanic or Latino 96%
Study Details
Setting
The study was conducted in 12 middle and high schools in Virginia.
Study sample
The study recruited secondary school teachers who expected to be the primary instructor of a course that had end-of-course standardized achievement exams. Participating teachers in each school were grouped by the subject (i.e., math/science, language arts/social studies) they would teach in the evaluation. Within each subject in each school, participating teachers were randomly assigned to either the MTP-S intervention or “business-as-usual” professional development. Teachers were then asked to select their “focal class” for inclusion in the evaluation—the class that they expected to be the most academically challenging and that had end-of-year achievement tests. Although initial selection of focal classes occurred before teachers learned their research condition, some teachers later changed their focal classes. Written parental consent and student assent were obtained for participating students in the focal classes after randomization. Because the teachers’ ultimate selection of focal classes, parental consent, and student assent could have been affected by knowledge of the teacher’s research condition, the study was determined by the WWC to have a compromised random assignment process. The study included 78 teachers (40 MTP-S and 38 comparison) from 12 schools. The analytic sample for the intervention year included 1,267 students (606 MTP-S and 661 comparison) taught by 76 teachers (39 MTP-S and 37 comparison). The analytic sample for the post-intervention year was composed of a new cohort of 970 students (419 MTP-S and 551 comparison) who had not participated in the study during the previous year. These post-intervention year students were taught by 61 teachers (27 MTP-S and 34 comparison) who had participated in the study during the intervention year; however, the MTP-S teachers did not receive MTP-S coaching during the post-intervention year. The mean grade of the students across both cohorts was grade 8. Forty-six percent of students were female, and 29% had families with incomes at less than 200% of the poverty line. The racial/ethnic demographics were as follows: 72% were White, 22% were Black, 4% were Hispanic, 2% were Asian, and 1% were of other race/ethnicity.
Intervention Group
Intervention group teachers attended an initial workshop, where consultants defined the MTP-S principles and described the dimensions of high-quality teacher–student interactions from the CLASS-S. Approximately twice a month throughout the school year, each teacher submitted videotaped sessions from his or her focal class to a consultant, who identified brief segments of the session for review and discussion. The teacher reviewed the identified segments for his or her own behaviors and for student reactions and answered questions from the consultant about the connection between the behaviors and the reactions. The teacher then conferred with the consultant in a 20- to 30-minute discussion by phone in which the consultant recommended strategies to enhance teacher–student interactions. Consultants also directed teachers to view annotated video exemplars of high-quality teaching available on the MTP-S website. The year of coaching was followed by a brief “booster” workshop. The intervention involved a total of about 20 hours of in-service training over 13 months. The mean years of teaching experience was 7.6 during the intervention year. Sixty-nine percent of MTP-S teachers were female, and 59% held a master’s or higher level degree during the intervention year.
Comparison Group
Comparison group teachers received regular in-service training. They videotaped six classroom segments throughout the school year at times that coincided with MTP-S teacher videotapings, but they did not receive feedback on the videotapes. Comparison teachers did not have access to the MTP-S library of video exemplars. The mean years of teaching experience was 10.1 during the intervention year. Fifty-seven percent of comparison teachers were female, and 70% held a master’s or higher level degree during the intervention year.
Outcome descriptions
An outcome in the general achievement domain was reported. Student achievement was measured using end-of-year scores from tests taken in core subjects, with the score on the most comparable course in the previous school year used as a pretest. Because the authors report a single test score without distinguishing academic achievement in specific subject areas, the outcome falls in the general achievement domain. For a more detailed description of this outcome measure, see Appendix B.
Support for implementation
Prior to the beginning of the school year, both MTP-S and comparison group teachers participated in a 3-hour workshop in which they received instruction in procedures for videotaping classes and submitting the videotapes (as well as procedures for study-related tasks, such as obtaining student assent/parent consent and collecting self-reported data). Consultants for the MTP-S teachers were master teachers trained in using the CLASS-S system. Two consultants led an initial workshop for intervention teachers that outlined MTP-S principles and discussed the CLASS-S dimensions of high-quality teacher–student interactions.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Allen, J., Gregory, A., Mikami, A., Lun, J., Hamre, B., & Pianta, R. (2013). Observations of effective teacher-student interactions in secondary school classrooms: Predicting student achievement with the Classroom Assessment Scoring System–Secondary. School Psychology Review, 42(1), 76–98.
-
Hafen, C. A., Allen, J. P., Mikami, A. Y., Gregory, A., Hamre, B., & Pianta, R. C. (2012). The pivotal role of autonomy in secondary school classrooms. Journal of Youth and Adolescence, 41, 245–255.
-
Mikami, A. Y., Gregory, A., Allen, J. P., Pianta, R. C., & Lun, J. (2011). Effects of a teacher professional development intervention on peer relationships in secondary classrooms. School Psychology Review, 40(3), 367–385.
-
Hafen, C. A., Allen, J. P., Gregory, A., Mikami, A. Y., Hamre, B., & Pianta, R. C. (2012, November). Improving teaching quality in secondary schools through professional development: Evidence from two RCT’s of the My Teaching Partner program [Study 1]. Paper presented at Association for Public Policy Analysis and Management Conference, Baltimore, MD. Retrieved from https://appam.confex.com/appam/2012/webprogram/
An Interaction-Based Approach to Enhancing Secondary School Instruction and Student Achievement
Review Details
Reviewed: February 2012
- Quick Review (490 KB) (findings for My Teaching Partner-Secondary (MPT-S))
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
---|---|---|---|---|---|---|---|---|---|
Standards of Learning (SOL) |
My Teaching Partner-Secondary (MPT-S) vs. Business as usual |
Intervention Year |
Overall;
|
460.30 |
465.60 |
No |
-- | ||
Show Supplemental Findings | |||||||||
Standards of Learning (SOL) |
My Teaching Partner-Secondary (MPT-S) vs. Business as usual |
Post-intervention year |
NULL;
|
488.20 |
482.20 |
Yes |
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 46%
Male: 54% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Virginia
-
Race Asian 2% Black 21% Other or unknown 1% White 72% -
Ethnicity Hispanic 4% Not Hispanic or Latino 96%
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).