Project Activities
CEDR is housed at the University of Washington and is directed by Dan Goldhaber (PI). Its mission is to conduct independent education research that provides policymakers and practitioners the scientific evidence necessary to make decisions that improve education outcomes. CEDR's research work generally addresses the following areas of education policy with the intent to facilitate evidence-based policy and practice: school and teacher effectiveness, education accountability and governance, and teacher labor markets. Empirical work at CEDR focuses on the analysis of longitudinal data that associates teachers with individual students. CEDR has currently partnered with 11 Washington State school districts through SDPI, including Spokane Public Schools.
SPS is the largest school district in eastern Washington, and the second largest in the state. The district consists of 34 elementary, 6 middle, and 5 high schools. In the 2012-13 school year, the district employed 1,758 teachers who instruct 28,768 students. During the previous 5 years, Spokane has hired 988 new teachers from a pool of 5,840 applicants. The project involved the Employment Services Department of Spokane Human Resources. Employment Services, directed by Angela Jones (Co-PI), oversees the school district's recruitment, screening, and hiring processes, including the assessment of protocols used during the hiring process, and the training of administrators responsible for hiring. The Department is also responsible for the maintenance of the administrative datasets that will be used to conduct the proposed analyses.
This study built on an existing researcher-practitioner partnership, formed under CEDR's Strategic District Partnership Initiative. The partnership was strengthened by increasing Spokane's capacity to approach educational interventions through data analysis, and by the framework that was formed to guide the project. The partnership assembled an Advisory Board of researcher and practitioner experts, and at several points convened meetings of Spokane stakeholders to discuss data, methods of analysis and interpretation of results, as well as to obtain feedback. The partnership studied the relation between information collected from teacher applicants and later teacher performance.
Structured Abstract
Setting
This study was conducted onsite in Spokane and at the Center for Education Data and Research in Seattle.
Sample
The project used data from local and state records on teacher applicants to Spokane (from SY 2007–08 through 2012–13) and their students.
District hiring could be an important policy lever with the potential to significantly influence the effectiveness of the teaching workforce in improving student academic outcomes. However, the relation between applicant data and future teacher performance is not well understood. Specifically, empirical findings regarding the relation between teacher credentials (e.g., licensure, certification, graduating institution, degree/major) and later student outcomes are mixed and at times contradictory. Previous analyses, however, have focused on data integrated into the state longitudinal databases and have not included information about teacher applicants collected and recorded by some local school systems. Moreover, previous analyses have not included the observations of local hiring officials and these observations of prospective teacher characteristics might be important predictors of teacher effectiveness in the future.
Initial research
They collected data generated by Spokane Public Schools online applicant management system, applicant scores on screening instruments used by the district, and state-wide administrative data including standardized test-based measures of teacher performance, attrition, absences, and licensure exam scores.
Data analytic strategy
The analysis used data about teacher applicants combined with information drawn from statewide longitudinal datasets from Washington State, which included data on student and teacher performance. Combining these datasets allowed the inclusion of the performance of applicants who were not hired by Spokane but instead were hired by other school districts in the State.
These research aims were addressed in four steps: (1) assessment of variation in applicant data collected; (2) assessment of variation in all variables among applicant subgroups defined by progress through the hiring process; (3) assessment of variation in independent variables among teachers of varying performance level; and (4) multivariate regression analysis of the link between applicant data teacher quality.
Additionally, they performed an analysis of inter-rater reliability of the screening instrument used to determine which applicants would receive.
Key outcomes
This partnership produced an article in a peer-reviewed journal, a presentation, and a IES funded exploration project (R305A170060). The findings of this project, which informed decision-making in Spokane Public Schools, include the following:
- Two of the screening instruments used by SPS were predictive of value-added measures of teacher performance and attrition but not absences
- Soliciting letters of recommendation directly from references promoted greater candor in the recommendation
- Applicant screening forms used by SPS tallied by hand had an 18.8% error rate
- There is low inter-rater reliability of the 60-point screening instrument used by the district to determine which applicants progress to interviews
People and institutions involved
IES program contact(s)
Products and publications
ERIC Citations:Find available citations in ERIC for this award here.
Journal articles
Goldhaber, D., Grout, C., & Huntington-Klein, N. (2017). Screen twice, cut once: Assessing the predictive validity of teacher selection tools.Education Finance and Policy, 12(2): 197-223. Full text
Martinková, P., Goldhaber, D., & Erosheva, E. (2018). Disparities in ratings of internal and external applicants: A case for model-based inter-rater reliability. PloS ONE, 13(10), e0203002. doi.org/10.1371/journal.pone.0203002 Full text
Related projects
Supplemental information
Co-Principal Investigator: Jones, Angela
Partner: Spokane Public Schools
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
