REL Midwest Ask A REL Response

Educator Effectiveness

February 2019

Banilower, E. R., Heck, D. J., & Weiss, I. R. (2007). Can professional development make the vision of the standards a reality? The impact of the National Science Foundation’s Local Systemic Change through Teacher Enhancement Initiative. Journal of Research in Science Teaching, 44(3), 375–395. Retrieved from https://eric.ed.gov/?id=EJ760189

From the ERIC abstract: “Professional development is seen as one of the major levers for aligning science instruction in the USA with the vision put forth by national standards documents. Although there is a growing consensus regarding what constitutes effective professional development, there is little empirical evidence to support this consensus. This study examines the impact of professional development that is content-based, situated in classroom practice, and sustained over time on teacher attitudes, perceptions of preparedness, and classroom practices. It utilizes longitudinal data from the National Science Foundation’s Local Systemic Change through Teacher Enhancement Initiative (LSC), collected from 42 projects over a span of 7 years. The professional development model used in the LSCs differed from previous initiatives in that it targeted all teachers in a jurisdiction and emphasized preparing teachers to implement project-designated instructional materials. Analyses of the data provide evidence that this model for professional development has an impact on teachers and their classroom practices. In addition, the analyses found that teachers’ perception of principal support for ‘Standards’-based science instruction is an important predictor of these outcomes.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Darling-Hammond, L., & Richardson, N. (2009). Research review/teacher learning: What matters?. Educational Leadership, 66(5), 46–53. Retrieved from http://www.ascd.org/publications/educational-leadership/feb09/vol66/num05/toc.aspx

From the abstract: “Research shows how schools can create more powerful professional development experiences. To help young people learn the more complex and analytical skills they need for the 21st century, teachers must learn to teach in ways that develop higher-order thinking and performance. To develop the sophisticated teaching required for this mission, education systems must offer more effective professional learning than has traditionally been available. What does research say about the kind of professional learning opportunities that improve instruction and student achievement?”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Garet, M. S., Heppen, J. B., Walters, K., Parkinson, J., Smith, T. M., Song, M., et al. (2016). Focusing on mathematical knowledge: The impact of content-intensive teacher professional development (NCEE 2016-4010). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://eric.ed.gov/?id=ED569154

From the ERIC abstract: “This report examines the impact of content-intensive Professional Development (PD) on teachers’ math content knowledge, their instructional practice, and their students’ achievement. The study’s PD had three components, totaling 93 hours. The core of the PD was ‘Intel Math,’ an intensive 80-hour workshop delivered in summer 2013 that focused on deepening teachers’ knowledge of grades K-8 mathematics. Two additional PD components totaling 13 hours were delivered during the 2013-14 school year: the ‘Mathematics Learning Community,’ a series of five 2-hour collaborative meetings focused on analyzing student work; and ‘Video Feedback Cycles,’ a series of three one-on-one coaching sessions where teachers’ lessons were observed and critiqued. The purpose of these two components was to reinforce the math content in Intel Math and help teachers apply the content to improve their instruction. Grade 4 teachers from 94 schools in six districts and five states participated in the study and were randomly assigned within schools to either a treatment group that received the study PD or a control group that did not receive the study PD. The key findings on the impact of the study PD on teacher knowledge, practice, and student achievement include: (1) The PD had a positive impact on teacher knowledge; (2) The PD had a positive impact on some aspects of instructional practice, particularly ‘Richness of Mathematics’; and (3) Despite the PD’s generally positive impact on teacher outcomes, the PD did not have a positive impact on student achievement. The study then addressed these research questions: (1) Was the study PD implemented with fidelity; (2) What were the features of the PD as implemented; (3) To what extent did teachers participate in the PD; and (4) What was the impact on teachers’ content knowledge, teachers’ classroom practices, and student achievement, of offering content-focused PD relative to business-as-usual PD? The results show that the study PD did change some aspects of teachers’ knowledge and classroom practice, but not in a way that led to improved student achievement. This may be partially explained by the finding that the math content knowledge and dimensions of instructional practice targeted by the study PD were generally not correlated with student math achievement. The one exception was ‘Errors and Imprecision,’ on which the study PD did not have a statistically significantly impact. Thus, future research might focus on identifying PD that will improve this aspect of practice. Future research might also seek to identify other aspects of knowledge and practice to target with PD that are more strongly related to improved student achievement.”

Garet, M. S., Heppen, J., Walters, K., Smith, T., & Yang, R. (2016). Does content-focused teacher professional development work? Findings from three Institute of Education Sciences studies. Washington, DC: American Institutes for Research. Retrieved from https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=NCEE20174010

From the description: “Subject knowledge is widely viewed as important for teaching, and professional development (PD) often aims to build such knowledge. This brief synthesizes findings from three large-scale random assignment studies of PD that were conducted by the National Center for Education Evaluation and Regional Assistance in the Institute of Education Sciences. Although the PD programs in each study were different, they all emphasized building teachers’ content knowledge or knowledge about content-specific pedagogy. The programs combined summer institutes with periodic teacher meetings and coaching during the school year. These programs were compared to the substantially less intensive PD that teachers typically received in study districts. The studies found that the PD boosted teachers’ subject knowledge and some aspects of instructional quality, but did not have a positive impact on student achievement. The studies also found that most of the measured aspects of teachers’ knowledge and practice were not correlated with student achievement. This consistent pattern of findings suggests that future studies should seek to better understand on what aspects of teacher knowledge and practice PD should focus, and how PD can achieve a larger impact on this knowledge and practice.”

Glover, T. A., Nugent, G. C., Chumney, F. L., Ihlo, T., Shapiro, E. S., Guard, K., et al. (2016). Investigating rural teachers’ professional development, instructional knowledge, and classroom practice. Journal of Research in Rural Education, 31(3). Retrieved from https://eric.ed.gov/?id=EJ1101917

From the ERIC abstract: “Teachers Speak was a national survey study designed to investigate the characteristics of rural elementary school teachers’ existing professional development; differences in professional development practices between rural and non-rural settings; and the potential influence of professional development characteristics on rural teachers’ knowledge, perceptions, and instructional practice. The respondents included 268 rural and 327 non-rural (city, suburban, town) teachers whose schools were selected via stratified random sampling. Key findings indicate that professional development experiences, perceptions, and classroom practices were similar for rural and non-rural teachers. Rural teachers did not appear to be comparatively disadvantaged, at least not in terms of their best professional development experiences. They reported comparable characteristics for professional development (e.g., providers, hours, practice and feedback opportunities, collaboration opportunities). An emphasis on topics during professional development was found to be related to increased (a) positive perceptions of the utility of the topics, (b) perceptions of knowledge gained pertaining to those topics, and (c) increased focus on those topics during classroom instruction. Perceived utility of instructional topics was a significant predictor of reported practice. When including both rural and non-rural teachers, time in professional development was found to be a significant predictor of their pedagogical content knowledge.”

Heck, D. J., Banilower, E. R., Weiss, I. R., & Rosenberg, S. L. (2008). Studying the effects of professional development: The case of the NSF’s Local Systemic Change Through Teacher Enhancement Initiative. Journal for Research in Mathematics Education, 39(2), 113–152. Retrieved from https://eric.ed.gov/?id=EJ786728

From the ERIC abstract: “Enacting the vision of NCTM’s ‘Principles and Standards for School Mathematics’ depends on effective teacher professional development. This 7-year study of 48 projects in the National Science Foundation’s Local Systemic Change Through Teacher Enhancement Initiative investigates the relationship between professional development and teachers’ attitudes, preparedness, and classroom practices in mathematics. These programs included many features considered to characterize effective professional development: content focus, extensive and sustained duration, and connection to practice and to influences on teachers’ practice. Results provide evidence of positive impact on teacher-reported attitudes toward, preparedness for, and practice of ‘Standards’-based teaching, despite the fact that many teachers did not participate in professional development to the extent intended. Teachers’ perception of their principals’ support for ‘Standards’-based mathematics instruction was also positively related to these outcomes.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Heller, J. I. (2012). Effects of Making Sense of SCIENCE™ professional development on the achievement of middle school students, including English language learners (Final Report. NCEE 2012-4002). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://eric.ed.gov/?id=ED530414

From the ERIC abstract: “This study evaluated an approach to professional development for middle school science teachers by closely examining one grade 8 course that embodies that approach. Using a cluster-randomized experimental design, the study tested the effectiveness of the Making Sense of SCIENCE professional development course on force and motion (Daehler, Shinohara, and Folsom 2011) by comparing outcomes for students of teachers who took the course with outcomes for students of control group of teachers who received only the typical professional development offered in their schools and districts. The study estimated impacts on student science achievement for all grade 8 students in the study sample as well as for the subsample of English language learners. It also estimated impacts on teacher science and pedagogical knowledge. Results for the primary confirmatory analyses indicate that after adjusting for multiple comparisons, there were no statistically significant differences between the test results on science content of students in intervention group classrooms and students in control group classrooms. Intervention group students in neither the full sample (effect size = 0.11) nor the English language learner subsample (effect size = 0.31) scored significantly higher on the ATLAST Test of Force and Motion than did their control group counterparts. Similarly, intervention group students in neither the full sample (effect size = 0.03) nor the English language learner subsample (effect size =0 .09) scored higher on the physical science reporting clusters of the California Standards Test than did their control group counterparts. Results for the intermediate confirmatory analyses indicate that after adjusting for multiple comparisons, teachers who received the professional development course outscored their control group counterparts on the ATLAST Test of Force and Motion for Teachers (effect size = 0.38), as well as on their ratings of confidence in their ability to teach force and motion (effect size = 0.49). With one exception, the study findings were not sensitive to variations in specification of the estimation models. The exception is that, for teacher content knowledge, inclusion of the pretest in the impact analysis model (basic model plus pretest) decreased the point estimate from 9.8 to 6.1 and the effect size from 0.61 to 0.38. In exploratory analyses, the study investigated whether there were differential impacts on student and teacher content knowledge outcomes across the six research sites. The estimated impacts were most pronounced at two of the six sites. For the full sample of students, point estimates for student and teacher content knowledge of force and motion followed exactly the same rank order at all sites. There are three main limitations of this study. First, there was high sample attrition: 48 of the 181 teachers who were randomly assigned to intervention and control groups left the study before data collection was completed. However, there is no evidence that attrition resulted in significant differences at the baseline between the intervention and control samples used in the analysis. Second, the study did not include analyses of classroom implementation of course-related practices. As a result it is not possible to infer whether the lack of student effects is due to a failure of treatment group teachers to modify classroom practices or a failure of modified practices to affect student outcomes. Third, the findings are based on volunteer teachers and students whose parents provided consent. It is possible that the findings would have been different had teachers been required to participate in the intervention, and all students been tested.”

Kang, H. S., Cha, J., & Ha, B. W. (2013). What should we consider in teachers’ professional development impact studies? Based on the conceptual framework of Desimone. Creative Education, 4(04), 11. Retrieved from https://www.scirp.org/Journal/PaperInformation.aspx?PaperID=30903

From the abstract: “Teacher professional development has long been of interest since it affects teachers’ learning, the practice of teaching, and student learning. In reality, as substantial resources have been spent on professional development, policy makers increase their search for evidence-based research about its effects on teachers’ and students’ outcomes. Therefore, it is imperative to use a solid framework evaluating professional development. Specifically, Desimone (2009) provides a comprehensive framework for evaluating the effect of professional development. Specifically, she represents that: 1) core features of effective professional development are content focused, active learning, coherence, duration, and collective participation; 2) the way this effective professional development affect teachers’ knowledge, their practice, and finally students’ learning; and 3) contextual factors such as student characteristics, teacher characteristics, and school characteristics are related to the effectiveness of professional development. Through this study, Desimone’s framework has been supported by both theoretical literature and empirical studies. Furthermore, some implications were provided for policy makers and school leaders as well as for teachers in Korea.”

Rotermund, S., DeRoche, J., & Ottem, R. (2017). Teacher professional development by selected teacher and school characteristics: 2011-12 (Stats in Brief. NCES 2017-200). Jessup, MD: National Center for Education Statistics. Retrieved from https://eric.ed.gov/?id=ED573871

From the ERIC abstract: “This Statistics in Brief provides a snapshot of the state of teacher professional development activities among U.S. public school teachers using data collected through the 2011–12 Schools and Staffing Survey (SASS) Public School Teacher Questionnaire. This report relies on data provided by public school teachers about their professional development activities during the 2011–12 school year. The report focuses on public school teachers’ responses to questions regarding the topics covered in their professional development activities; the amount of time spent in those activities in the last 12 months; the support they received for participation; and whether they engaged in less formal professional activities, such as working collaboratively with other teachers on instruction. The report examines each of these aspects of public school teachers’ professional development not only nationwide but also by the level of the schools in which they taught, their years of teaching experience, and the locale in which they taught.”

Shakman, K., Zweig, J., Bocala, C., Lacireno-Paquet, N., & Bailey, J. (2016). Teacher evaluation and professional learning: Lessons from early implementation in a large urban district. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Northeast & Islands. Retrieved from https://eric.ed.gov/?id=ED565633

From the ERIC abstract: “Policymakers and researchers increasingly recommend aligning educator evaluations and professional development to improve instruction and student learning. However, few empirical studies have examined the relationship between new educator evaluation systems and the professional development in which teachers engage following their evaluation. This study looked closely at one large urban district’s educator evaluation system from May 2013 to May 2014. The study examined the written feedback evaluators provided to teachers who were rated less than proficient in one or more standards of effective teaching practice: curriculum, planning, and assessment (standard 1); teaching all students (standard 2); family and community engagement (standard 3); and professional culture (standard 4). The data for this study consisted of teacher characteristics and evaluation ratings, prescriptions, a district-administered teacher survey in which teachers reported on their professional activities related to each standard, and a small number of teacher and principal interviews. The following are the key findings from the study: (1) Teachers received prescriptions across all four standards, usually for one or two professional activities per prescription, and they received more prescriptions with professional practice activities than with professional development activities; (2) Teachers reported participating in more professional activities, including both professional development and professional practice activities, for instruction-based standards (standards 1 and 2) than for non-instruction-based standards (standards 3 and 4); (3) For all standards, less than 40 percent of teachers who responded to the survey participated in all the activities their evaluators prescribed. However, at least 75 percent of teachers who received a prescription for standard 1 (curriculum, planning, and assessment) or standard 2 (teaching all students) and responded to the survey reported participating in at least one professional activity that related to those standards. For standards 3 and 4, fewer teachers engaged in the prescribed activities, but many engaged in other types of professional activities, including professional development or professional practice activities, related to the standard; and (4) Of the teachers rated less than proficient who had received a prescription for standard 1 and then participated in any professional activities related to that standard, 64 percent received at least a proficient rating on a subsequent evaluation; 34 percent of the teachers with prescriptions for standard 1 who did not participate in related activities also raised their summative rating to proficient. Standard 1 was the only standard for which a statistically significant difference was detected in the subsequent evaluation of teachers who engaged in activities aligned to their prescriptions and those who did not. The percentage of teachers in the study group who received at least a proficient rating on their subsequent evaluation did not vary by whether they participated in the particular type of activity their evaluator prescribed.”

Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. L. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers. REL 2007-No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest. Retrieved from https://eric.ed.gov/?id=ED498548

From the ERIC abstract: “The Regional Educational Laboratory - Southwest (REL Southwest) conducted a systematic and comprehensive review of the research-based evidence on the effects of professional development (PD) on growth in student achievement in three core academic subjects (reading/ELA, mathematics, and science). The primary goal of this study was to address the question, What is the impact of teacher participation in professional development on student achievement? Nine studies emerged as meeting What Works Clearinghouse (WWC) evidence standards, from more than 1,300 manuscripts identified as potentially relevant. Although the number of studies that met evidence standards was small, the average overall effect size of 0.54 was observed when examined within the three content areas included in the review. The consistency of this effect size indicates that across all forms and content of PD, providing training to elementary school teachers does have a moderate effect on their students’ achievement. However, because the average number of contact hours averaged almost 49 hours across the nine studies, the total contact hours must be substantial to get such an effect size. Because of the limited number of studies and the variability in the PD that was represented among the nine studies we examined, we were unable to make any conclusions about the effectiveness of specific PD programs or about the effectiveness of PD by form, content, or intensity.”