Project Activities
Researchers will use administrative data from local and state records to explore how professional reference ratings of the performance of teaching applicants correlate with later teaching and student outcomes. Then, researchers will randomly assign hiring personnel to receive or not receive the professional reference ratings to examine extent to which the provision of PR ratings data influences hiring decisions.
Structured Abstract
Setting
This project will take place in schools located in a large district consisting of urban, suburban, and rural communities in eastern Washington State.
Sample
The project will use administrative data from local and state records on more than 3000 teaching applicants for over 600 teaching positions (from SY 2015-16 through 2019-20) and (if hired within the State) their students.
Malleable factors are professional reference ratings of the performance of teaching applicants, applicant type (e.g., novice vs. experienced), job type (e.g., elementary vs. middle school), and PR type (e.g., principal vs. university professor). These factors could lead to the development of a tool that could be reliably and efficiently used by any school district.
Research design and methods
This study will employ a correlational research design to examine the predictive validity of PRs' assessments of applicants and potential moderation by applicant type (e.g., novice vs. experienced), job type (e.g., elementary vs. middle school), and PR type (e.g., principal vs. university professor). A randomized controlled research design will be used to examine extent to which the provision of PR ratings data influences hiring decisions.
Control condition
In the control condition, the applicant online profile will not contain PR ratings and instead contain a "censored report" with the following text: "For research purposes Professional Reference ratings have been blinded for this applicant. Which applicants have blinded results is determined at random and is not indicative of the quality of the applicant."
Key measures
Primary predictor measures include applicant background information (e.g., degree, GPA, professional experience), PR ratings, and applicant scores on a 60-point rubric used by district hiring officials to screen applicants. Outcome measures include value-added modeling scores based on student achievement scores in mathematics and reading, observational evaluation scores on the Marzano-based rubric, and teacher retention at the school, district, and state levels.
Data analytic strategy
Researchers will conduct descriptive analyses of the correlations and distributions of PR ratings, applicant characteristics, PR characteristics, job characteristics, and teacher outcomes. Researchers will also conduct exploratory factor analysis of the PR ratings data to understand the latent construct of the data and the relationship between observed ratings and latent factors. The relation between the PR ratings data and each teacher outcome will be examined using multivariate regression analysis. To examine the relation between PR ratings and applicant scores researchers will use regression analysis in the context of random provision of applicant data to hiring officials.
People and institutions involved
IES program contact(s)
Products and publications
Researchers will produce preliminary evidence of potentially promising teaching candidate screening procedures and peer-reviewed publications.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications
Journal articles
Goldhaber, D., Grout, C., Wolff, M., & Martinková, P. (2021). Evidence on the dimensionality and reliability of professional references' ratings of teacher applicants. Economics of Education Review, 83, 102130.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
