Project Activities
Researchers seek to answer four research questions (RQs) about licensure tests:
- RQ1: What are existing relationships between teacher content knowledge in specific domains—as measured by MTEL subarea and objective scores—and student achievement, and how do these relationships vary by subject and grade level?
- RQ2: What are existing relationships between teacher content knowledge in specific domains—as measured by MTEL subarea and objective scores—and teacher summative ratings, and how do these relationships vary by subject and grade level?
- RQ3: To what extent can the relationships between MTEL scale scores and student and teacher outcomes (described by RQs 1 and 2) be strengthened by differential weights on particular MTEL test objectives, subareas, or sections?
- RQ4: To what extent does changing the weights on different subareas and objectives affect the licensure test performance of different teacher subgroups and the predicted passing rates of these teacher subgroups?
Answers to RQs 1 and 2 will provide empirical evidence on aspects of teacher content knowledge that are predictive of teacher effectiveness (value-added to test scores in RQ1 and teacher evaluations in RQ2). The exploratory evidence for RQs 1 and 2 provides a foundation for the most novel component of the proposal: assessing the extent to which reweighting specific domains of teacher content knowledge might strengthen the connection between licensure test performance and instructional effectiveness (RQ3). We additionally plan to provide evidence based on historical data about how different licensure test policies might impact the composition of candidates who meet the state's licensure requirements (RQ4).
Structured Abstract
Setting
The setting of the proposed research is the state of Massachusetts, which has the 17th largest public school system in the country and currently has about 1 million students and 70,000 teachers in its public schools. Massachusetts requires applicants for a prekindergarten (PreK)–12 educator license to pass MTEL tests in Communication and Literacy Skills (Reading and Writing) and in at least one additional PreK–12 academic subject area when available.
Sample
The primary sample will consist of all 73,558 teacher candidates who took at least one MTEL test between 2009 and 2016, as well as their future students in K-12 public schools.
This project will focus on applicant content knowledge and weights on subgroup of the Massachusetts licensure test items. Each test consists of one to three sections (multiple choice, short answer, and/or open response) that include different objectives (broad subject matter knowledge domains) arranged within different subareas (particularized tasks).
Research design and methods
Researchers will use ordinary least squares (OLS), logit, constrained linear regression, regularized regression to determine weights and predictive validity of the Massachusetts license test sub-scale.
Control condition
Due to the nature of the study, there is no control condition.
Key measures
The key measure of interest is MTEL licensure test performance on the various items, objectives, subareas, and sections of each test. Outcomes include student performance on the state's standardized tests and the summative performance rating of teachers under the state's evaluation framework.
Data analytic strategy
For RQ1 and RQ2, researchers estimate associations between licensure test domain and subarea scores and teacher outcomes using regression models common in the literature that control for student, classroom, and school context. For RQ3, researchers plan to first estimate individual teacher performance measures using the summative ratings and student achievement data, and then estimate optimal predictions of these performance measures using the MTEL testing data. Using the standardized teacher fixed effects and the MTEL objective scores, researchers will estimate optimal prediction weights via constrained linear regression. Given concerns about the precision of these estimates, researchers plan to supplement these analyses with regularization methods that penalize large weights on any one subsection. Finally, RQ4 will examine changes in the distribution of scale scores across candidate subgroups (e.g., by race and ethnicity) in the historical data according to these different reweighting schemes.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Research activities will result in policy memos and presentations to the Massachusetts Department of Elementary and Secondary Education, policy presentations to researchers, and research publications.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
