|Title:||RESET: Recognizing Effective Special Education Teachers|
|Principal Investigator:||Johnson, Evelyn||Awardee:||Boise State University|
|Program:||Professional Development for Teachers and School-Based Service Providers [Program Details]|
|Award Period:||4 years (7/1/2015 – 6/30/2019)||Award Amount:||$1,588,173|
Purpose: The purpose of this project is to develop and validate a special education teacher observation measure, Recognizing Effective Special Education Teachers (RESET), designed to evaluate and improve instructional practice delivered to students with disabilities. The challenges of evaluating special education teachers are significant. Special educators work under a variety of conditions, serve a heterogeneous group of students with disabilities, enter the profession with varying skill levels, and may require additional instruction to meet the needs of struggling learners. These factors establish a need for an evaluation system that will help lead to high-quality, evidence-based instructional techniques focused on improving outcomes of students with disabilities in a variety of teaching contexts. This study aims to meet this need through RESET, an evaluation tool intended to gain a greater understanding of the characteristics, processes, and outcomes that are associated with high-quality instructional practices for student with disabilities.
Project Activities: Using evaluation criteria the research team previously developed, the RESET observation tool and accompanying user manual will be prepared for testing. The team will collect approximately 500 hours of video of teachers' instruction to be used to determine the reliability and validity of RESET. Twenty-five teachers not included in the videos will be recruited, trained, and asked to rate the video data according to RESET criteria. The data obtained from the teacher ratings of the video will be used to finalize the RESET tool.
Products: The products of this project will include evidence of validity and reliability for RESET use with special educators, as well as peer-reviewed publications and presentations.
Setting: The research will take place in two school districts in Idaho.
Sample: The study sample will include 150 special education teachers of students with high-incidence disabilities in kindergarten through 12th grade. The research team will work with special education directors to identify teachers who serve the identified population of students, group teachers by grade level and then by instructional context (i.e., self-contained classroom), and draw a random sample of teachers from within each group to participate. An additional 25 teachers with at least 5 years of teaching experience and a graduate degree in special education will be recruited to rate the videos.
Assessment: The assessment being examined is the special education teacher observation tool, RESET, which is based on the theory of action that the quality of instruction that a teacher provides is a key determinant of a student's individual growth. The foundation of RESET is derived from Charlotte Danielson's Framework for Teaching (FFT), with a focus on the domain of instruction. RESET will evaluate common features of sound instructional practice and will delineate criteria for evaluating evidence-based instructional practices appropriate for students with disabilities. Through a computerized, evaluation system, RESET will rely on the use of video capture of instruction for trained observers to evaluate the quality of the instruction using criteria based on prior research. Teachers evaluated by RESET will receive targeted feedback from the tool itself to promote effective instructional practices in their classrooms.
Research Design and Methods: The research team will collect approximately 500 hours of recorded teachers' instruction from 150 special education teachers. Building on prior research, the researchers will iteratively develop criteria for the evidence-based instructional practices on which teachers will be evaluated. Twenty-five teachers will be recruited for data coding to determine the reliability of RESET using the following steps: 1) teachers will be trained on rating the videos to determine inter-rater reliability; 2) ratings will be used in a generalizability study to help determine the ideal number of observations required per video and whether performance improves over time with feedback from trainers; and 3) data will be analyzed to determine the internal consistency estimates of component subscales. Researchers will use Michael Kane's argument-based validation model with four related sets of inferences – scoring, generalization, extrapolation, and implication. These inferences will be tested empirically to examine whether RESET can reliably determine the special education teachers with the most effective instructional practices; measure and provide targeted, specific, corrective feedback for teacher instructional practice; and link student growth rates to effective teaching practices.
Control Condition: Due to the nature of the research design, there is no control condition.
Key Measures: RESET, in its current form, will be used to observe and evaluate special education teachers. Ratings from trained teachers and project staff will be used to assess the reliability and validity of the evaluation tool. Pretest and posttest data of students will be collected to examine student growth rates over the year.
Data Analytic Strategy: Intra-class correlations will be used to assess inter-rater reliability for RESET. Generalizability theory analyses (i.e., multilevel analyses, reliability coefficients) will be used to determine the optimum number of observations per teacher. Internal consistency reliabilities will be computed to determine consistency of the subscale ratings. An examination of the distribution of scores, inter-rater agreement analysis, and confirmatory factor analysis of a random sample of observations will be used to assess how appropriate the RESET criteria will be in discriminating between more or less effective instructional practices. Correlations will be conducted to assess how teacher evaluation and student growth scores relate. Qualitative data from a technical advisory group will inform the consistency in the interpretation of feedback teachers will receive.
Johnson, E. S. & Beymer, L. L. (2016). Special education teacher candidate evaluation: Creating a preservice to master teacher observation system. In J. Goeke & K. Kossars Special Education Teacher Preparation, 325T OSEP Program.
Journal article, monograph, or newsletter
Johnson, E. S., Ford, J. W., Crawford, A., & Moylan, L. A. (2016). Issues in Evaluating Special Education Teachers: Challenges and Current Perspectives. Texas Education Review, 4(1): 71–83. Retrieved from https://repositories.lib.utexas.edu/bitstream/handle/2152/45928/Johnson_Issue_Evaluating_Special_Ed_Teachers_TxEdRe.pdf?sequence=1. Full text