Ask A REL Response

June 2020

Question

What research has been conducted on the efficacy of assessment measures used as admission and/or certification criteria for teachers; and, the differential impact on people of color?

Response

Following an established REL Southeast research protocol, we conducted a search for research reports as well as descriptive study articles on the efficacy of assessment measures used as admission and/or certification criteria for teachers; and, the differential impact on people of color. We focused on identifying resources that specifically addressed the efficacy of assessment measures used as admission and/or certification criteria for teachers; and, the differential impact on people of color. The sources included ERIC and other federally funded databases and organizations, research institutions, academic research databases, and general Internet search engines (For details, please see the methods section at the end of this memo.)

We have not evaluated the quality of references and the resources provided in this response. We offer them only for your reference. These references are listed in alphabetical order, not necessarily in order of relevance. Also, we searched the references in the response from the most commonly used resources of research, but they are not comprehensive and other relevant references and resources may exist.

Research References

1. Aravena, F., & Quiroga, M. (2018). Certification policy: Reflections based on the Chilean case of the INICIA test for beginner teachers. Center for Educational Policy Studies Journal, 8(4), 29-46. http://eric.ed.gov/?id=EJ1201215
   From the abstract: "The main purpose of this paper is to analyse a specific educational policy in a national context: INICIA (In Spanish: Start) in Chile. Enacted in 2008, this policy evaluates beginning teachers at the national level before they start their professional careers in schools. The INICIA has been categorised as a certification policy to measure what teachers know in relation to a certain disciplinary area. At present, INICIA is voluntary for beginning teachers. However, due to policy changes, a passing score on it will soon be necessary to become a nationally qualified teacher in Chile. Through a holistic analysis of the policy, we develop a complex picture of the problem INICIA generates as a requirement for certification. According to our analysis, INICIA has been misunderstood as a policy that provides quantitative and qualitative information about a teacher's performance. In actuality, INICIA merely provides information about a specific moment in a teacher's professional development and cannot be used as a predictor of future performance."
2. Bae, J. (2020). The impact of the edTPA on visual arts teacher education in Wisconsin. Studies in Art Education: A Journal of Issues and Research in Art Education, 61(1) 64-84. http://eric.ed.gov/?id=EJ1241504
   From the abstract: "This study concerns the nationally available, high-stakes, performance-based assessment for teacher candidates called the Educative Teacher Performance Assessment (edTPA) and its impact on visual arts teacher education in Wisconsin, where it is required for certification. Participants in this case study were 10 visual arts teacher educators at Wisconsin public universities who had taught art methods courses for art education majors and who had supervised teacher candidates in the edTPA era. Qualitative data were collected via online surveys and follow-up Skype interviews, and content was analyzed using a phenomenological method. The results showed that all participants had incorporated preparation for the edTPA in their art teacher education programs, but they also had had a variety of experiences and attitudes regarding the value of the assessment. The findings point to some tensions around the use of the edTPA as a measure of teacher readiness. Recommendations for research and policy are included."
3. Greenblatt, D. (2018). Neoliberalism and teacher certification. Policy Futures in Education, 16(6) 804-827. http://eric.ed.gov/?id=EJ1190514
   From the abstract: "The dominant discourse of teacher education is framed as the "construction of the problem of teacher education" (Cochran-Smith et al., 2013). From this neoliberal discourse came the push for more accountability, contributing to the promotion of a nationally scored standardized teacher performance assessment (edTPA). This article examines the impact of the assessment on those taking the elementary education edTPA in New York City, teacher educators and the teacher education curriculum. Based on survey and interview data from teacher candidates and faculty, there is a strong concern about the impact of the cost of the edTPA on economically disadvantaged teacher candidates and under-resourced schools of education. In many cases, teacher candidates were not encouraged to fight the neoliberal discourse but rather to "play the game" to pass a test. Because teacher educators felt it was their job to help their teacher candidates get certified, little resistance was found even among those who were not in support of the edTPA. In resistance to the standardization, teacher candidates were encouraged to be strategic, within the Pearson guidelines, in terms of their class size and composition as well as with the lessons that they taught. There were also accommodations made in teacher candidates placements to avoid testing grades or schools with a standardized curriculum; however, this potentially created new consequences with the avoidance of high-needs schools. Additionally, due to the intersection of the specific requirements of the edTPA and the mandated curricula, teacher candidates went outside the guidelines to pass the test. Furthermore, some teacher candidates decided not to take the edTPA and to get certified in another state or not to go into public education."
4. Heafner, T., & Petty, T. (2016). Using edTPA to compare online and face to face teacher preparation programs. Journal of Technology and Teacher Education, 24(2), 153-186. http://eric.ed.gov/?id=EJ1112484
   From the abstract: "Central to determining the effectiveness of technology to support learning and the value of technology-mediated instruction is the quality of programs. edTPA is a widely accepted, national measure of teacher readiness and preparation. Using edTPA score reports for teacher candidates completing a teacher education program, this study provides data informed evidence of the equivalency of online teacher candidates' learning outcomes as compared to candidates completing traditional face to face (F2F) programs. Mean and summative performance results suggest that mode of course delivery was not a significant factor in preparing teacher candidates. In general, learning was comparable in either format; however, analyses at the element level offer evidence of differences in achievement outcomes associated with program structure. These findings challenge notions that traditional methods of teacher preparation are the only reputable pathways to becoming a teacher and suggest attributes of online learning may be linked to differences in candidate knowledge and skills preparation when equitable curricular opportunities are provided. Based on the results of this study, authors contend that not only is online an equivalent and in some cases more effective method for preparing teachers; it may be a more adept program delivery platform for preparing educators for planning for instruction and assessing student learning outcomes."
5. Hildebrandt, S. A., & Swanson, P. (2019). Re-examining teacher candidate performance on world language edTPA as a consequential assessment. NECTFL Review, 84, 15-33. http://eric.ed.gov/?id=EJ1245560
   From the abstract: "Used by over 800 educator preparation programs in more than 40 states and the District of Columbia, edTPA seeks to measure content-specific outcomes of beginning teachers as a high-stakes assessment. Many states also use edTPA to inform teacher licensure or certification decisions, making the assessment consequential for individual teacher candidates. The present study focuses on teacher candidates' performance on the World Language edTPA, targeting two world language teacher education programs since the assessment became consequential for licensure in their respective states. It then compares findings to those of a previous study of the same programs' World Language edTPA performance (Hildebrandt & Swanson, 2014) from before the assessment became consequential for teacher licensure decisions in 2015 in Illinois and Georgia. Programmatic changes were implemented after initial inspection of the data, and results from such curricular changes are discussed in terms of local impact. Additionally, national trends are identified and discussed. Findings demonstrate serious issues regarding this assessment of beginning teacher effectiveness as related to World Language teaching and learning. This research has implications for teacher preparation programs and other educational stakeholders alike."
6. Mahoney, M. P. (2015). Pearson-praxis assessments review teacher certification assessment technology education: A report for the council on technology and engineering teacher education. Journal of Technology Education, 27(1), 78-89. http://eric.ed.gov/?id=EJ1088031
   From the abstract: "The Praxis Series, developed by Educational Testing Services (ETS), has been the long standing assessment for teacher licensure. It is comprised of three separate skills examinations: (1) Praxis Core Academic Skills for Educators (Core) (attempts to measure academic skills in reading, writing, and mathematics; content knowledge of candidates entering teacher preparation programs); (2) Praxis I Pre-Professional Skills Tests (PPST) (attempts to measure basic skills in reading, writing, and mathematics; used to qualify candidates for entry into a teacher education program); and (3) Praxis II Subject Assessments (attempts to measure subject-specific content knowledge, as well as general and subject-specific teaching skills, deemed necessary for beginning teaching). Since the early 2000s, Pearson Education began offering a series of updated teacher certification assessments. Pearson Education currently offers two categories of assessment tools for candidate teachers. These include the following: (1) National Evaluation Series (NES) (Entry-level assessment that attempts to reflect contemporary teacher knowledge and skill sets); and (2) Custom Programs (CP) (designed for specific content and to reflect individual state needs; edTPA (formerly the Teacher Performance Assessment developed at Stanford University) Performance-based assessment protocols developed to evaluate candidate teachers level of classroom preparation; owned and authored by Stanford University). In this article, Mark Mahoney compares the Praxis and Pearson assessments and their respective as well as combined usage across states in the US. Based on his analysis, Mahoney recommends the Council on Technology and Engineering Teacher Education (CTETE) consider further investigation into becoming an accrediting agency for teacher certification institutions or programs in technology and engineering education. He goes on to point out that CTETE would have to decide the degree of accreditation that they wish to address--national, state, or a combination of the two. Further, he states that the CTETE must establish a history of accreditation practices prior to applying for recognition. Partnering with other organizations (e.g., the International Technology and Engineering Educators Association, the American Society for Engineering Education, and the Association for Career and Technical Education) may provide a more comprehensive and secure foundation from which to build an accrediting agency."
7. Russell, V., & Davidson Devall, K. F. (2016). An examination of the edTPA portfolio assessment and other measures of teacher preparation and readiness. Foreign Language Annals, 49(3), 479-501. http://eric.ed.gov/?id=EJ1115036
   From the abstract: "The authors examined the outcomes on several measures of world language teacher preparedness, including university- and state-mandated summative evaluations and the edTPA portfolio assessment, for seven world language teacher candidates during their final semester of clinical practice. The candidates were enrolled in an initial certification program (Spanish P-12) at the same university in Georgia. The results revealed that edTPA scores were not well aligned with mentor teachers' and supervisors' evaluations on the university-wide instrument and that measures of teacher content knowledge and target language proficiency did not correlate well with candidates' edTPA scores. The disparity between edTPA scores and other measures of teacher preparation was most apparent for the two nonnative English speakers. The findings suggest that candidates whose primary language is not English may need additional support with academic English. Moreover, the present study found that candidates, mentors, and supervisors lacked an understanding of how the various university, state, and national assessments fit together to measure the knowledge, skills, and dispositions that are necessary for effectively teaching a world language. Training on the content and purpose of the edTPA may be needed so that stakeholders understand the benefit of having a national-level view of candidates' knowledge and skills."
8. Zhang, J., Wright, A. M., Kim, E-J., & Szilágyi, J. (2019). A collaborative journey toward inclusive teacher education programs. Curriculum and Teaching Dialogue, 21(1-2). 37- 51. http://eric.ed.gov/?id=EJ1247231
   From the abstract: "The journey described in this article reflects the ongoing collaboration between two faculty in a Childhood Inclusive Teacher Education Program seeking to further develop a more inclusive curriculum. Crosswalk examples between the Council for Accreditation of Educator Preparation (CAEP) and Council for Exceptional Children (CEC) standards and two course assessment examples are included to help provide insights into the process of creating meaningful assessments for use in a dualcertification teacher education program."
9. Zhou, M. (2018). edTPA as a tool to measure teacher readiness: A case study on first year teachers. Georgia Educational Researcher, 14(2), 51-68. http://eric.ed.gov/?id=EJ1194570
   From the abstract: "The case study intends to study edTPA as a measure for teacher readiness. The case study examined four first year teachers' edTPA experience, performance, and their perspectives on edTPA as a tool to measure teacher readiness in light of their first year teaching experiences and TKES assessment results. Three research questions were used to guide the study: What were first year teachers' edTPA experience and performance? What were first year teachers' TKES assessment results? What were first year teachers' perspectives on edTPA as a tool to measure teacher readiness? Four first year teachers in north GA participated in the study. These participants passed edTPA assessment prior to employment as full time teachers. The participants were interviewed individually. Six interview questions were used. Interviews were structured for about 45 min each. Data were recorded, transcribed, and coded for analysis to see trends, patterns, and themes. The results of the study indicated that edTPA did not measure all aspects of teacher readiness. Factors such as diversity, relationship building, collaboration were valued by participants as important for teacher readiness in the classroom. edTPA as an effectiveness tool to measure teacher readiness was uncertain. Discussion and future implications of the research results on edTPA to teacher preparation, teacher quality, and policy making were offered."

Methods

Keywords and Search Strings
The following keywords and search strings were used to search the reference databases and other sources:

• Assessment measures, certification criteria for teachers
• Teacher certification, assessment measures

Databases and Resources
We searched ERIC for relevant resources. ERIC is a free online library of over 1.6 million citations of education research sponsored by the Institute of Education Sciences. Additionally, we searched Google Scholar and PsychInfo.

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

• Date of the publication: References and resources published for last 15 years, from 2003 to present, were include in the search and review.
• Search Priorities of Reference Sources: Search priority is given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations, academic databases, including ERIC, EBSCO databases, JSTOR database, PsychInfo, PsychArticle, and Google Scholar.
• Methodology: Following methodological priorities/considerations were given in the review and selection of the references: (a) study types - randomized control trials,, quasi experiments, surveys, descriptive data analyses, literature reviews, policy briefs, etc., generally in this order (b) target population, samples (representativeness of the target population, sample size, volunteered or randomly selected, etc.), study duration, etc. (c) limitations, generalizability of the findings and conclusions, etc.

This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Southeast Region (Alabama, Florida, Georgia, Mississippi, North Carolina, and South Carolina), which is served by the Regional Educational Laboratory Southeast at Florida State University. This memorandum was prepared by REL Southeast under a contract with the U.S. Department of Education's Institute of Education Sciences (IES), Contract ED-IES-17-C-0011, administered by Florida State University. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.