Search Results: (1-15 of 23 records)
|REL 2021095||Examination of the Validity and Reliability of the Kansas Clinical Assessment Tool
Although national assessments for evaluating teacher candidates are available, some state education agencies and education preparation programs have developed their own assessments. These locally developed assessments are based on observations of teaching and other artifacts such as lesson plans and student assignments. However, local assessment developers often lack information about the validity and reliability of data collected with their assessments. The Council for the Accreditation of Educator Preparation (CAEP) has provided guidance for demonstrating the validity and reliability of locally developed teacher candidate assessments, yet few educator preparation programs have the capacity to generate this evidence.
The Regional Educational Laboratory Central partnered with educator preparation programs in Kansas to examine the validity and reliability of the Kansas Clinical Assessment Tool (K-CAT), a newly developed tool for assessing the performance of teacher candidates. The study was designed to align with CAEP guidance. The study found that cooperating teachers reported that the K-CAT accurately represented existing teaching performance standards (face validity). Two skilled raters found that the content of the K-CAT was mostly aligned to existing teaching performance standards (content validity). In addition, K-CAT scores for the same teacher candidate, provided by cooperating teachers and supervising faculty, were positively related (convergent validity). K-CAT indicator scores showed internal consistency, or correlations among related indicators, for standards and for the tool overall (reliability). K-CAT scores showed small relationships with teacher candidate scores on other measures of teaching performance (criterion-related validity).
|REL 2021056||Exploring Teachers’ Influence on Student Success in an Online Biology Course
This study of an online high school biology course offered by Florida Virtual School examined the amount of variation in course completion, students’ final exam scores, and time to completion that is attributable to the influence of teachers. This study examined three different student outcomes for segment 1 of the course: the rate of course completion, score on the final exam at the end of the course segment, and time taken to complete the segment. Students' end-of-segment exam varied only slightly across teachers, but teachers showed more influence on completion rates and time to completion. As a result, students with the highest- and lowest-performing teachers had notable differences in their time to completion and minor differences in course completion and exam scores.
|REL 2020037||Teacher Turnover and Access to Effective Teachers in the School District of Philadelphia
Concerned about the expense of teacher turnover, its disruption to schools and students, and its potential effect on students' access to effective teachers, the School District of Philadelphia partnered with the Regional Educational Laboratory Mid-Atlantic to better understand students' access to effective teachers and the factors related to teacher turnover. This analysis of differences in teacher effectiveness between and within schools in the district found that teachers of economically disadvantaged, Black, and Hispanic students had lower evaluation scores than teachers of non–economically disadvantaged and White students but similar value-added scores (a measure of teacher effectiveness based on student academic growth). The study also found that each year from 2010/11 through 2016/17, an average of 25 percent of the district’s teachers left their school and 8 percent left the district. During the first five years of teaching, 77 percent of teachers left their school and 45 percent left the district. Turnover rates were highest for teachers who taught middle school grades, teachers who missed more than 10 days of school a year, teachers who identified as Black, teachers who had previously changed schools, and teachers who had low evaluation ratings. Teacher turnover was higher in schools where teachers had a less positive view of the school climate. School climate mattered more for teachers with higher evaluation ratings than for teachers with lower evaluation ratings.
|REL 2018274||The Texas Teacher Evaluation and Support System rubric: Properties and association with school characteristics
The purpose of this study was to examine the data from the 2014/15 pilot implementation of the Texas Teacher Evaluation and Support System (T-TESS) in order to understand certain properties of the T-TESS rubric, which consists of 16 dimensions classified within 4 domains of teacher effectiveness. The dataset included over 8000 teachers across 251 schools and 51 districts that participated in the pilot. Descriptive statistics were reviewed to assess the extent to which the T-TESS rubric ratings differentiate teacher effectiveness. Correlational analysis was performed to determine the internal consistency of the rubric. Uniqueness values, which resulted from a factor analysis of T-TESS’s 16 dimensions, were examined to determine whether each dimension makes some unique contribution. Lastly, regression analysis was conducted to explore the relationships between T-TESS performance ratings and school characteristics. Results indicate that in the 2014/15 pilot of T-TESS, 1.6 percent of teachers were rated as improvement needed, 24.9 percent as developing, 68.3 percent as proficient, 3.7 percent as accomplished, and 1.5 percent as distinguished based on the T-TESS rubric for teacher effectiveness. The T-TESS rubric is internally consistent at both the domain and dimension levels. All dimension-to-dimension within domains and domain-to-domain correlations are positive, suggesting that none of the domains or dimensions stand out as unrelated or contradictory to the rest of the system. Findings also suggest that the T-TESS rubric is efficient. None of the domains or dimensions are clearly redundant, as supported by findings that no correlation is close to one. In addition, an analysis of uniqueness reveals that each dimension makes some unique contribution. Although statistically significant relationships are found between observation ratings and school characteristics, the combination of observed student and school characteristics explains, at most, approximately 8 percent of the variation in overall observation ratings for high schools, and explains even less for elementary and middle schools. One area for future research is the validation of the ratings with other measures of teacher effectiveness, such as student growth. However, the validity of a teacher evaluation system itself may not necessarily translate into improvements in teacher effectiveness or into long-term outcomes, such as teacher retention and greater student achievement. Therefore, further research could explore whether the implementation of such systems do in fact relate to the more distal measures.
|REL 2017208||Stated Briefly: Patterns of classroom quality in Head Start and center-based early childhood education programs
This "Stated Briefly" report is a companion piece that summarizes the results of another report of the same name. REL researchers analyzed data from the 2002/03 Head Start Impact Study (U.S. Department of Health and Human Services) using latent class analysis to determine whether multiple measures, each designed to address one aspect of classroom quality, could collectively differentiate classrooms in a consistent and substantively meaningful way. Using data on measures such as structural quality, process quality, teacher-child interactions, and instructional activities, they found that Head Start (n = 1,061) and center-based (n = 421) classrooms may be grouped according to three classroom quality patterns: good, fair, and poor. The researchers also found that classroom quality measures determined by independent observers distinguish classroom quality groups better than self-reported measures.
There are three main implications of this study: (1) it is possible to use multiple dimensions of the classroom experience to identify classroom quality patterns; (2) identifying classroom quality patterns will likely require independent observers; and (3) an individual classroom may not be perfectly characterized by its classroom quality group. This exploratory study, which was supported by the Early Childhood Education Research Alliance, shows an alternative way to measure classroom quality and provides an example of what patterns of classroom quality exist in programs serving Head Start-eligible children across the country—thus informing practitioners about what quality looks like in these settings and adding to the literature regarding measuring quality in early childhood education. Practitioners and policymakers can use the results of this study to inform the way that they measure the quality of their classrooms and to examine further the characteristics and practices of the different groups of classrooms.
|NCEE 20174010||Does Content-Focused Teacher Professional Development Work? Findings from Three Institute of Education Sciences Studies
Subject knowledge is widely viewed as important for teaching, and professional development (PD) often aims to build such knowledge. This brief synthesizes findings from three large-scale random assignment studies of PD that were conducted by the National Center for Education Evaluation and Regional Assistance in the Institute of Education Sciences. Although the PD programs in each study were different, they all emphasized building teachers' content knowledge or knowledge about content-specific pedagogy. The programs combined summer institutes with periodic teacher meetings and coaching during the school year. These programs were compared to the substantially less intensive PD that teachers typically received in study districts. The studies found that the PD boosted teachers' subject knowledge and some aspects of instructional quality, but did not have a positive impact on student achievement. The studies also found that most of the measured aspects of teachers' knowledge and practice were not correlated with student achievement. This consistent pattern of findings suggests that future studies should seek to better understand on what aspects of teacher knowledge and practice PD should focus, and how PD can achieve a larger impact on this knowledge and practice.
|REL 2016143||Development and implementation of quality rating and improvement systems in Midwest Region states
Recent federal and state policies that recognize the benefits of high-quality early childhood education and care, such as the Race to the Top–Early Learning Challenge and the Preschool for All initiative, have led to a rapid expansion of quality rating and improvement systems (QRISs). Although 49 states implement a QRIS in some form, each system differs in its approach to defining, rating, supporting, and communicating program quality. This study examined QRISs in use across the Midwest Region to describe approaches that states use in developing and implementing a QRIS. The purpose was to create a resource for QRIS administrators to use as they refine their systems over time. Researchers used qualitative techniques, including a review of existing documents and semistructured interviews with state officials in the Midwest Region to document the unique and common approaches to QRIS implementation. Findings suggest that the process of applying for a Race to the Top–Early Learning Challenge grant helped advance the development of a QRIS system, even in states that were not awarded funding. Also, all seven states in the Midwest Region use a variety of direct observations in classrooms to measure quality within each QRIS, despite the logistical and financial burdens associated with observational assessment. Five of the states in the Midwest Region use alternate pathways to rate certain early childhood education programs in their QRIS, most commonly for accredited or state prekindergarten programs. Finally, linking state subsidies and other early childhood education funding to QRIS participation encouraged early childhood education providers to participate in a QRIS. Developing and refining a QRIS is an ongoing process for all states in the Midwest Region and systems are continually evolving. Ongoing changes require policymakers, researchers, providers, and families to periodically relearn the exact requirements of their QRISs, but if changes are based on evidence in the field of changing needs of children and families, revised QRISs may better measure quality and better serve the public. Findings from this report can help inform the decisions of state QRIS administrators as they expand and refine their systems.
|REL 2015030||Redesigning Teacher Evaluation: Lessons from a Pilot Implementation
REL Northeast and Islands, in collaboration with the Northeast Educator Effectiveness Research Alliance and the New Hampshire Department of Education, conducted a study of the implementation of new teacher evaluation systems in New Hampshire’s School Improvement Grant (SIG) schools. While the basic system features are similar across district plans, the specifics of these features vary considerably by district. District fidelity to the plans, as measured by the exposure of teachers to different features of the evaluation system, ranged from moderate to high. Researchers identified several factors related to implementation: capacity of administrators to conduct evaluations; initial and on-going evaluator training; the introduction and design of student learning objectives; and the professional climate of schools, including the support of the new system by teachers and evaluators.
|NCEE 20144001||Access to Effective Teaching for Disadvantaged Students
Recent federal initiatives emphasize measuring teacher effectiveness and ensuring that disadvantaged students have equal access to effective teachers. This study substantially broadens the existing evidence on access to effective teaching by examining access in 29 geographically dispersed school districts over the 2008-2009 to 2010-2011 school years.
The report describes disadvantaged students' access to effective teaching in grades 4 through 8 in English/language arts (ELA) and math, using value-added analysis to measure effective teaching. On average, disadvantaged students had less access to effective teaching in these districts. Providing equal access to effective teaching for FRL and non-FRL students would reduce the student achievement gap from 28 percentile points to 26 percentile points in ELA and from 26 percentile points to 24 percentile points in math in a given year.
|NCEE 20134014||Evaluation of the Regional Educational Laboratories: Interim Report
The ten Regional Educational Laboratories (RELs) respond to needs of states and school districts for applied research, technical assistance for use of data and research, and dissemination of research findings. The RELs' work is authorized by the Education Sciences Reform Act (ESRA) and is carried out under five-year contracts with the Institute of Education Sciences.
This interim report provides descriptive information on the activities undertaken by the RELs during the 2006-2011 REL contracts with IES. It also presents expert ratings of the technical quality and relevance of the RELs' short-term education research and technical assistance projects, known as Fast Response Projects.
|REL 2013002||Using Alternative Student Growth Measures for Evaluating Teacher Performance: What the Literature Says
States increasingly are interested in incorporating measures of student achievement growth in teacher evaluations. But the typical measure of student growth—progress on state assessments from one school year to the next—usually covers only reading and math and only in grades 4–8. Members of REL Mid-Atlantic's Teacher Evaluation Research Alliance wanted to understand more about the alternatives, and the REL produced this literature review in response. It examines the following two alternatives to state assessments: applying statistical value-added methods to outcomes other than student performance on standard state assessments and using student learning objectives (SLOs) developed by individual teachers.
|NCEE 20134015||The Effectiveness of Secondary Math Teachers from Teach For America and the Teaching Fellows Programs
Teach For America (TFA) and the Teaching Fellows programs are important and growing sources of teachers of hard-to-staff subjects in high-poverty schools, but comprehensive evidence of their effectiveness has been limited. A large-scale random assignment study examines the effectiveness of secondary math teachers from two highly selective alternative certification route programs: Teach for America (TFA) and Teaching Fellows.
The study separately compares the effectiveness of teachers from each program with the effectiveness of other teachers teaching the same subjects in the same schools. On average, students assigned to TFA teachers had higher math scores at the end of the school year than students assigned to comparison teachers. Students of Teaching Fellows and comparison teachers had similar math scores, on average. However, students with Teaching Fellows teachers did outperform students whose teachers entered the classroom through less selective alternative routes.
|NFES 2013802||Forum Guide to the Teacher-Student Data Link: A Technical Implementation
This publication is a practical guide for implementing a teacher-student data link (TSDL) that supports a range of uses at the local, regional, and state levels. The guide addresses the considerations for linking teacher and student data from multiple perspectives, including governance, policies, data components, business rules, system requirements, and practices. It provides references to promising practices for high quality data linkages, including TSDLspecific processes such as roster verification and the establishment of the Teacher of Record. The information and opinions published here are those of the Forum and do not necessarily represent the policy or views of the U.S. Department of Education or NCES.
|WWC SSR209||WWC Review of the Report "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood"
Published in 2011, The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood examined differences in student achievement when students were taught by a teacher with high or low value-added, a measure of teacher effectiveness. The study sample included about 3,300 cohorts of math and reading students in grades 4-8 in a large, urban school district from 1991 to 2009. The WWC review focuses on the report's analyses of what happened to students' achievement as a result of the movements of teachers into and out of schools with relatively high or low value-added estimates. Researchers analyzed standardized student math and reading assessment scores and found that when a high value-added teacher started to teach students in a cohort, or a low value-added teacher stopped teaching students in a cohort, the event was associated with a statistically significant increase in reading and math test scores. In addition, when a high value-added teacher stopped teaching students in a cohort, or a low value-added teacher started teaching students in a cohort, the event was associated with a statistically significant decrease in test scores. The cohorts of students being compared were well matched on test scores before an event occurred. However, the analyses for long term outcomes, such as college attendance and employment, had no distinct intervention and comparison groups, making them ineligible for WWC review. Therefore, the research analyzing math and reading scores described in this report meets WWC evidence standards with reservations.
|NCEE 20124049||Evaluation of the Quality Teaching for English Learners Program
For report NCEE 2012-4005 Evaluation of Quality Teaching for English Learners (QTEL) Professional Development http://ies.ed.gov/ncee/edlabs/projects/project.asp?ProjectID=88
This file contains data from a study which adds to the evidence base on effective strategies for teaching English language learner students, this report studies the impact on middle grades student achievement of a teacher professional development program.
The program, Quality Teaching for English Learners (QTEL), is an approach to improving the teaching of English language learner students. The program aims to enhance the ability of teachers to work with English language learner students and increase the quality of instruction for all other students in the mainstream classroom. QTEL summer institutes consist of seven days of professional development to provide a foundation for using new tools and processes for the academic and linguistic development of adolescent English language learner students.
This study, Evaluation of Quality Teaching for English Learners (QTEL) Professional Development, examined QTEL's effects on student outcomes in English language arts and English language development, as measured by the California Standards Test and the California English Language Development Test. No significant effects of QTEL were found on student achievement. For teachers, no significant effects were found on attitudes, knowledge, or practice, as measured by the teacher survey, teacher knowledge assessment, and a classroom observation protocol, respectively.
The study sample included middle schools in urban and suburban areas of three Southern California counties. The study was conducted from 2007-2010.