Project Activities
This study compared three versions of PD for the TELL curriculum with varying numbers of components. The research team initially conducted activities to prepare the intervention for use and recruit and train the first cohort of teachers. Following preparation activities, each of the three cohorts received the assigned PD activities and implemented TELL in their classrooms. Data were collected from all study participants during those years. In addition, an implementation study was completed with a subset of participants randomly selected from those teachers, and administrators, who participated in the research.
Structured Abstract
Setting
This research took place in pre-kindergarten programs operated by school districts in Arizona that include children eligible for education services for speech and language impairment under Part B of the Individuals with Disabilities Act (IDEA).
Sample
Participants in this study included 127 pre-kindergarten teachers who had experience teaching in a classroom with children with disabilities and 403 children (5 to 6 children per class). The children were in their final year of pre-kindergarten (primarily age 4) receiving IDEA Part B services for developmental speech and/or language impairment as a primary or secondary condition.
TELL is a class-wide curriculum with evidenced-based oral language and early literacy teaching practices that can be embedded into any preschool activity, such as small or large group instruction and outdoor play. Lesson plans are designed for embedding the practices into different content areas, including science, math, art, music, movement, and social-emotional development. In this study, researchers compared three variations of PD for implementing TELL with fidelity. In addition, some PD activities that had been implemented in-person in previous studies occurred online in this study. The standard PD group received face-to-face training during a workshop approximately 2 months prior to the start of the school year and an additional face-to-face training after the winter break. The ModPD condition received the standard training plus monthly participation in professional learning communities through videoconferencing. The MaxPD condition received the standard training, participated in the professional learning communities, and received 12 individual coaching sessions from project coaches. For coaching, teachers uploaded videos of their own implementation, the coaches provided annotated feedback on the videos, and they met to discuss the video via videoconferencing.
Research design and methods
The researchers used a randomized controlled trial with three different conditions, each with a varying number of PD components for the TELL intervention—standard PD, ModPD, and MaxPD. Random assignment occurred at the school level, with classrooms nested within schools. Initially, the research team prepared to implement the intervention for the RCT, including printing and packaging the curriculum, acquiring all the needed materials, preparing additional support materials, and transferring the PD content that had been delivered face-to-face to the web-based system. In addition, the research team recruited teachers for the first cohort and conducted the initial workshop. After preparation activities, the first cohort implemented the intervention in their classrooms, participated in their respective PD activities, and participated in data collection. In following years, the second and third cohorts participated and the researchers analyzed the data and engaged in dissemination activities. The researchers examined whether there are significant differences in children's language and literacy outcomes based on the number of PD components teachers receive, as well as whether child outcomes are affected by different types of implementation fidelity, teacher and classroom outcomes, and other child characteristics. As part of the implementation study, the research team collected additional survey and qualitative data to create different classroom implementation profiles, adjusted them based on individual teachers, and generated hypotheses about what drives implementation.
Control condition
Although all PD versions were compared to one another, the counterfactual condition included training that is typically provided by school districts when a new curriculum is adopted, and was referred to as Standard PD. This included the initial workshop (12 hours) and a follow-up session (4 hours).
Key measures
Child outcome measures included investigator-developed curriculum-based measures, which assess receptive and expressive vocabulary, letter names, letter sounds, beginning sound awareness, narrative listening comprehension, expository book comprehension, and print knowledge. Standardized assessments also were used to measure child outcomes, including the Clinical Evaluation of Language Fundamentals Preschool-2nd Edition (CELF-P2), the Preschool Print and Word Awareness Test, and the Preschool Early Literacy Indicators. Parents completed a family demographic questionnaire as well as the CELF-P2 parent questionnaire to provide information about their child's language and literacy development. Teacher measures included standardized surveys to assess knowledge, beliefs about preschool literacy instruction, and self-efficacy; qualitative data from interviews and portfolios; a demographic survey; and ratings of school district implementation support and usefulness of each of the TELL PD activities. The researcher-developed fidelity measure included frequency, dose, and quality of language and literacy instruction during TELL-structured learning opportunities, based on observation of classroom videotapes submitted by teachers. Overall classroom quality was assessed with the Classroom Assessment Scoring System (CLASS).
Data analytic strategy
For the primary comparisons of outcomes between PD versions, the researchers used a linear mixed-modeling analysis of covariance (ANCOVA) and growth curve modeling. To examine whether differences in children's outcomes were affected by different aspects of teacher implementation, they used multi-level models. To examine what child, instructional, or teacher characteristics may moderate or mediate the impact of the intervention on child outcomes, they used interaction terms within the ANCOVAs. They examined differences in classroom quality and teacher implementation fidelity using multivariate ANCOVA. Teacher factors—such as self-efficacy and literacy instruction beliefs—were investigated with multivariate regression models. Implementation supports preferred by and most effective for teachers were described by integrating qualitative and quantitative data and creating classroom implementation profiles.
Cost analysis strategy
A cost analysis and cost-effectiveness analysis were conducted and compared across each version of the PD intervention. Cost-effectiveness was investigated by generating benefit-to-cost ratios for all three conditions.
Key outcomes
The main findings of this project, as reported by the principal investigator, are as follows:
- The primary outcome for this PD study was teacher implementation fidelity for the TELL curriculum. Results indicated no differences in teacher fidelity across the three conditions. All teachers improved in fidelity across the year of their participation in the research.
- Children’s proximal outcomes, as measured by the TELL curriculum-based measures, did not differ across the three PD conditions. All children improved from pre- to post-testing, irrespective of teacher PD condition.
- Children’s distal outcomes, as measured by standard assessments, did not differ across the three PD conditions. Again, all children improved from pre- to post-testing, irrespective of teacher PD condition.
- Teacher implementation fidelity was not observed to impact children’s proximal or distal outcomes.
- Classroom quality, as measured by the CLASS, revealed improvements from the beginning to the end of the school year, but there were no differences observed when comparing the teacher PD conditions.
- Key findings from the implementation science study revealed that all teachers felt equally supported in each of the PD conditions.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Project website:
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Available data:
The dataset from this study will be available on Open Science Framework when the final results are published.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
