Project Activities
Researchers will use teaching performance and student achievement data from administrative data systems as well as student surveys to estimate the impact of teachers being assigned PGPs on the outcomes of students and teachers in K–12 public and charter schools in New Mexico. The analysis will be performed using a "fuzzy" Regression Discontinuity (RD) design. Assuming that teachers (and determinants of student outcomes) immediately above and below the cutoff score for mandatory PGP assignment are similar to one another, researchers will compare the outcomes of teachers rated immediately above the cutoff to those of teachers rated immediately below the cutoff. Researchers will conduct complementary analyses to examine the impact of being rated "minimally effective" (separate from the PGPs) and PGP dosage (e.g., proportion of PGP completed; number of PGP elements; and supports received by frequency, location, and mode of delivery). Researchers will also conduct psychometric analysis of the NMTEACH measures, an implementation analysis and a cost analysis of the PGP policy.
Structured Abstract
Setting
This project will take place in K–12 public and charter schools in throughout the State of New Mexico.
Sample
The sample will include 21,000 educators who taught students in grades K-12 in the 2015-2016 to 2019-2020 school years. These teachers taught approximately 335,000 students per year, the majority of whom were Hispanic and from low-income households.
Intervention
The proposed study is based on New Mexico's teacher evaluation system called NMTEACH. The three primary components of this system are:
- tests of student achievement improvement such as value-added scores on state and end-of-course exams (50% of the overall evaluation);
- state-developed teacher observation rubric (25% of the evaluation);
- locally selected multiple measures from a menu of options approved by the state such as teacher attendance or student surveys (25% of the evaluation).
Research design and methods
Using a Regression Discontinuity (RD) design, researchers will analyze teaching performance and student achievement data from administrative data systems as well as student surveys to estimate the impact of teachers being assigned PGPs on the outcomes of teachers and students. A "sharp" RD design would not be appropriate in this context, because some teachers with ratings of "effective" or higher may be placed on a PGP due to district-level policy for individual ratings on a classroom observation score. However, being labeled "ineffective" or "minimally effective" substantially increases the chances of a teacher being assigned a PGP. Therefore, researchers will use a "fuzzy" design. Assuming that teachers (and determinants of student outcomes) immediately above and below the cutoff score for mandatory PGP assignment are similar to one another, researchers will compare the outcomes of teachers rated immediately above the cutoff to those of teachers rated immediately below the cutoff. Researchers will also examine implementation of the policy throughout the state using district personnel interviews and teacher surveys, conduct psychometric analyses of the NMTEACH measures, conduct supplementary analyses related to the PGP policy, and conduct a cost analysis.
Control condition
In the comparison condition, teachers are not labeled "ineffective" or "minimally effective" are not mandated to have professional growth plans.
Key measures
Student education outcome measures include the Partnership for Assessment of Readiness for College and Careers (PARCC) and end-of-course exams in math, ELA, science and social studies. Other measures include academic behaviors, student engagement and mindset as measured by validated survey instruments. Teacher measures include summative ratings on the NMTEACH Educator Effectiveness system. Researchers will measure the implementation of the intervention using district personnel interviews and teacher surveys.
Data analytic strategy
The primary strategy is based on a "fuzzy" RD design. Researchers will conduct additional regression discontinuity and GAMM analyses to provide context on the influence of labeling and dosage on PGP effects. Researchers will also examine the role of moderating variables, such as the role of opportunities for professional development in PGPs impact on student outcomes.
People and institutions involved
IES program contact(s)
Products and publications
Products: Researchers will produce evidence of the efficacy of mandatory professional growth plan assignments for teachers and students in K-12 public and charter schools in New Mexico (NM). This project will also result in peer-reviewed publications.
Supplemental information
Co-Principal Investigator: Montano, Matt; Mariano, Louis; Schwartz, Heather Schwartz; Schweig, Jonathan; Goldschmidt, Pete
Partner Institution: New Mexico Public Education Department
The components are weighted and combined to produce a summative evaluation score. Using this score, teachers are placed into five effectiveness categories—"ineffective," "minimally effective," "effective," "highly effective," and "exemplary". Teachers labeled "ineffective" or "minimally effective" are required to have professional growth plans (PGPs).
The mandatory PGP policy requires the teacher and formal evaluator to complete a document listing the areas of unsatisfactory performance, the goals for improvement, specific and measurable outcomes that will be monitored over the course of the review period, the timeline and frequency for tracking outcomes, and the resources that the teacher is suggested to consult to improve their practice. Outcomes are tracked for 90 days, and the teacher is expected to show improvement.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
