Project Activities
This project iteratively developed an online PD system using data on teachers' RTI knowledge and practice, teacher PD use, field-based observations of classrooms and data teams, and information on the usability and feasibility of PD modules. The research team conducted a randomized control trial to evaluate the feasibility and potential efficacy of the intervention for improving teachers' RTI implementation and, ultimately, students’ reading skills.
Structured Abstract
Setting
The research took place in elementary schools in Oregon and Washington. Additional teachers in states across the U.S. participated in field testing a measure of teachers’ RTI knowledge and skills in the first year of the project.
Sample
Five experienced elementary teachers were chosen for a Lead Development Team to provide consultation for the first 2 years of the project. One hundred third- through fifth-grade teachers from districts across the U.S. were recruited to field test a measure of RTI knowledge and skills. For the development and pilot testing of the intervention, 15 schools, 60 teachers, and 4,134 of their students participated in a sequence of randomized control trials.
The project resulted in an online professional development program that is integrated within the widely used easyCBM assessment system and provides teachers with training on sufficient student screening and progress measurement, developing instructionally adequate programs, and making data-based decisions for instructional change. The PD program includes 14 training modules focused on 3 key components of RTI identified in the literature: (1) measurement sufficiency (including the type of measure being used, the grade level of measures, and the frequency of measurement), (2) instructional adequacy (including tier, amount of time, grouping arrangement, curriculum materials and supplements, teaching strategies, social environments, and independent practice), and (3) data-based decision making (including interpreting progress-monitoring data to prompt changes in instruction relative to the level, variability, and amount of improvement in student performance). Each module includes a pre-test, instruction (easyCBM's illustrations of research-based best practices), and a post-test of proficiency.
Research design and methods
In phase 1, the research team worked closely with a Lead Development Team to iteratively develop and refine the PD modules and three measures of RTI practice (teacher survey of RTI knowledge and practice, observational protocol of teacher RTI implementation, and teacher usability and feasibility survey). The Lead Development Team reviewed initial drafts of PD modules and measures and provided feedback. Measures were field tested online by teachers across the U.S. and data were used to establish initial technical adequacy and to inform further refinement. In phase 2, the research team gathered data on feasibility and usability of the PD modules through an iterative process of implementation and comparisons, as well as consultation with the Lead Development Team. Specifically, they first compared Standard PD (teachers completed all 14 modules in numerical order at their own pace over the school year) to a business-as-usual control condition. Next, they compared Standard PD to Individualized PD (teachers were prompted to complete one or more of the 14 PD modules throughout the year based on their data management/use and student reading data) and Standard + Individualized PD (2 years of PD in which teachers completed all modules in their first year and received prompts to review specific topics only when interactions with the easyCBM indicated it would be helpful in their second year). They then tested the intervention with the best results in a larger randomized control trial pilot study.
Control condition
Control group teachers participated in business-as-usual RTI assessment systems within their school, with the understanding that they would receive the professional development intervention as they moved from delayed-treatment control group status to active treatment group status.
Key measures
The research team developed and tested measures of teacher RTI knowledge and practice, teacher assessment and instruction-related practices associated with RTI implementation, and PD module usability and feasibility. Data on teacher use (such as time on website, number of modules completed, time taken to complete each module, scores on pre- and post-proficiency tests) were collected automatically from the online delivery system and used to assess the extent to which PD was delivered as designed. Student reading outcomes were assessed by measures of reading automatically collected through the easyCBM assessment system, including passage reading fluency, vocabulary, and comprehension. Teacher implementation of RTI practices were measured by qualitative analysis of the appropriateness of their instructional groupings, interventions, and use of progress monitoring measures (both type and frequency) as documented within the easyCBM system.
Data analytic strategy
The research team analyzed the internal consistency of the teacher test of RTI knowledge and practice, bivariate correlations between test items, and construct validity using exploratory factor analysis. Feedback forms completed by the Lead Development Team on initial PD modules were qualitatively coded to identify recurring patterns and themes for further refinement of the PD program. The team used item response theory to analyze feedback collected from the module-specific usability and feasibility surveys. Multilevel growth models (with time nested within teacher, and teacher nested within condition) were used to evaluate the effects of the PD program on teacher RTI knowledge and practice and RTI implementation. The research team examined effects for all students and tested whether the impact varied for students with disabilities.
Key outcomes
The main findings of this project, as reported by the principal investigator, are as follows:
- Teachers who participated in the intervention significantly outperformed teachers in the control condition on a test of RTI knowledge and application at post-test, demonstrated more effective use of universal screening assessment results for grouping students for instruction, and demonstrated more frequent use of progress monitoring assessment results to inform discussions in individualized education program and student study team meetings and the intervention log available on easyCBM.
- Teachers who completed the PD reported being more comfortable explaining student assessment results to parents and colleagues and more confident in their decisions related to curriculum, grouping, and instructional strategies selected to meet individual student needs.
- Teachers maintained their increased knowledge of RTI as well as their improved RTI practice.
- Teacher outcomes did not differ significantly based on whether they were provided the Standard or Individualized PD.
- Although both treatment approaches were equally as effective at changing teachers’ practices and knowledge about RTI, participating teachers were unanimous in their advocacy for providing all 14 units of PD to all educators, irrespective of their prior knowledge about RTI. Discussions in all teacher focus groups revolved around the ways in which all of the PD units had been useful in either teaching new information or reminding the teachers about important information that they had learned before and that they appreciated learning again.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
