Co-Principal Investigators: Algina, James; Hemmeter, Mary Louise; McLean, Mary

Purpose: The goal of this study was to examine the efficacy of Tools for Teachers (TfT), a professional development intervention for using embedded instruction practices with preschool children with disabilities. Embedded instruction involves intentional teaching of individual child learning goals within the context of routine classroom activities. Prior research has found evidence that this approach can be effective with young children with varying disabilities, but embedded instruction practices are not always implemented with fidelity. Therefore, the research team created this professional development intervention to support teachers in implementing embedded instruction with children. This study used a cluster randomized controlled trial to compare the efficacy of two versions of the TfTprofessional development intervention—using on-site practice-based coaching (PBC) versus web-based PBC self-coaching—to one another and a business-as-usual group to determine the impacts on teacher practices and child outcomes.

Project Activities: The research team used a randomized controlled trial to test the efficacy of TfTwith data collected in four waves throughout the implementation year and three waves during a sustainability year. Data were analyzed to examine the impact of each TfT intervention on teacher and child outcomes, determine factors that may mediate or moderate the impacts, and the investigate the sustainability of teachers using embedded instruction practices over time.

Key Outcomes and Findings: The main findings of this project, as reported by the principal investigator, are as follows:

• Teachers in both TfT PD intervention conditions wrote higher quality embedded instruction learning targets than teachers in the BAU condition. Teachers in the TfT PD intervention with on-site coaching implemented more frequent and accurate embedded instruction learning trials than teachers in either the TfT PD self-coaching or BAU conditions. No differences in classroom emotional support, instructional support, or classroom organization were found across the three conditions at the end of the intervention year.
• In terms of sustainability of teacher practices, teachers in both TfT PD intervention conditions showed changes in their classroom organization practices from the end of the intervention school year to the end of the sustainability school year that were noteworthy as indicated by effect sizes. Teachers in both TfTconditions also generally maintained the quality of their written embedded instruction learning targets during the sustainability year. Noteworthy treatment effects for implementation of embedded instruction learning trials for teachers in the TfT on-site coaching condition continued to be evidenced at the end of their sustainability year relative to BAU teachers.
• Children whose teachers were in the TfT intervention condition with on-site coaching had better pre-academic outcomes—language, early literacy, and basic concepts—than children whose teachers were in the self-coaching TfT intervention or BAU conditions. There were no differences among the three groups of children on social-emotional or adaptive behavior outcomes. Treatment effects were not mediated by the quality of written embedded instruction learning targets or teachers' implementation of embedded instruction learning trials.
• The impact of TfT was moderated by child attendance in class. For children with higher attendance rates compared to other children in their class who were enrolled in the study, those whose teachers were in the on-site coaching condition performed better in language, literacy, and basic concept skills than those whose teachers were in the self-coaching or BAU conditions. Teacher certification also moderated the impact on children's early literacy. Children whose teachers received TfT PD with on-site coaching and had certification in early childhood education, general education, or special education achieved higher early literacy scores than children whose teachers received on-site coaching and had certification in early childhood special education.
• Teachers in both TfT PD intervention conditions found the interventions and embedded instruction for early learning feasible, acceptable, and useful. The on-site coaching condition was rated higher on utility and social validity than the intervention that included self-coaching.
• TfT workshops and on-site coaching were generally higher in total cost than for the BAU workshops and coaching, though costs per person and per contact hour of TfT workshops and on-site coaching were comparable or lower than those for BAU workshops and coaching. TfT self-coaching total, per person, and per hour costs were lower than BAU coaching; however, given the treatment effects for this condition, self-coaching is not recommended as the only type of coaching for teachers who are acquiring embedded instruction practices.

Structured Abstract

Setting: The research took place in preschool classrooms in rural, urban, and suburban areas in Florida, Tennessee, and Kentucky.

Sample: The researchers recruited 111 certified or licensed teachers in early childhood and special education classrooms and up to three eligible children (age 3 and 4 years) with developmental delays or disabilities from each classroom for a total of 327 preschoolers.

Intervention: Tools for Teachers (TfT) is a professional development intervention for using embedded instruction with preschool children with disabilities. The embedded instruction involved 14 key teaching practices organized by what to teach (specifying target skills aligned with children's IEP goals that would be taught in typically occurring classroom activities), when to teach (identifying appropriate activities in which to embed instruction), how to teach (using intentional and systematic instruction during ongoing activities), and how to evaluate (using data-based decision making to evaluate embedded instruction implementation and impacts on child learning). The four components of TfT professional development include (1) manualized workshops with demonstrations and opportunities to practice embedded instruction practices, (2) teacher implementation guides and materials, (3) a multi-media website with videos and more in-depth information, and (4) practice-based coaching. Coaching involved assessing teacher strengths and needs, setting goals and developing action plans, observing, reflecting, and providing feedback about embedded instruction implementation. The two intervention groups in this study received the coaching component differently. The first group received on-site coaching, with 16 weekly interactions between a teacher and project-trained coach. The second group engaged in self-coaching, with access to a separate section of the website with materials and supports for self-coaching and weekly emails to prompt self-coaching activities.

Research Design and Methods: The research team conducted a cluster randomized controlled trial, with randomization at the school level. Two cohorts of teachers were assigned to the TfT professional development with on-site coaching, TfT with self-coaching, or a business-as-usual control group. Classrooms/teachers participated for 2 years—the intervention year and sustainability year. In the first year of the study, the intervention was implemented along with four waves of data collection from before the intervention began to the end of the academic year. Data were collected on child, classroom, and teacher variables. In the second year, there were three waves of data collection on classroom and teacher variables. The primary aims of this efficacy trial were to examine the effects of the two TfT conditions relative to the business-as-usual group on teacher implementation of embedded instruction, classroom quality, and child outcomes (learning target behaviors, engagement, functional skills, pre-academic learning, and social and behavioral skills); confirm that TfT with on-site coaching produces greater impacts than with self-coaching; determine what other differences between groups emerged on key variables; demonstrate that both TfT interventions can be implemented with fidelity; and describe the conditions needed to support fidelity of implementation.

Control Condition: The business-as-usual control condition received their typical professional development activities and implemented their typical instructional practices.

Key Measures: The primary classroom outcome measures in this study included Classroom Assessment Scoring System (classroom instructional quality) and two measures of implementation of embedded instruction practices developed by the research team (Learning Targets Rating Scale, Embedded Instruction Observation System). Primary child outcome measures included the Embedded Instruction Observation System (EIOS, measures occurrence of child learning target behaviors), Engagement Behavior Observation System, Preschool and Kindergarten Behavior Scales-II, Preschool Language Scale-5, Test of Early Reading Ability-3, Bracken Basic Concept Scale-3 Receptive, and Adaptive Behavior Assessment System-3. Observations of classroom quality, teacher and parent questionnaires, school records, and a rating scale of child functional abilities were used to measure mediators and moderators as well as study inclusion criteria.

Data Analytic Strategy: Data were analyzed primarily with multivariate analysis of covariance models to determine group differences in teacher and child outcomes. Models for investigating treatment effects focused on intervention group averages that were adjusted for pre-intervention differences among the intervention groups. Mediation analyses focused on whether there were intervention effects on mediator variables and whether such effects were transmitted to outcome variables. Moderation analyses focused on whether intervention effects varied across sub-groups of children or teachers defined by variables measured pre-intervention. Change analysis for the sustainability year focused on differences among means over time. Statistical models accounted for effects of teachers and schools on child outcomes and schools on teacher outcomes.

Related IES Projects: Impact of Professional Development on Preschool Teachers' Use of Embedded-Instruction Practices (R324A070008); Tools for Families (R324A200044)

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Publicly available data: Data will be made publicly available in Open ICPSR in 2024. For additional information about how people can learn more about the data that will be made publicly available, contact patriciasnyder@coe.ufl.edu or algina@ufl.edu.

Project Website: http://embeddedinstruction.net/

Additional online resources and information: https://ceecs.education.ufl.edu/atoz/3rs/ https://ceecs.education.ufl.edu/letter-e-everyday-learning-opportunities/ https://ceecs.education.ufl.edu/letter-i-inclusion/ https://ceecs.education.ufl.edu/p-practice-based-coaching/

Select Publications

Journal articles

Shannon, D. K., Snyder, P. A., Hemmeter, M. L., McLean, M. (2022). Exploring coach-teacher interactions within a practice-based coaching partnership. Topics in Early Childhood Special Education, 40(4), 229–240.