Co-Principal Investigators: Greenwood, Charles; Carta, Judith; Schnitz, Alana

Purpose: The goal of this project is to validate a classroom observational measure that can be used to assist preschool teachers in revising and implementing interventions for children who are identified as unresponsive to existing classroom learning contexts (e.g., classroom activity structure such as small groups, academic content such as literacy). When a child's behavior is not responsive to the instruction provided, intervention adjustments are necessary. The Classroom Code for Interactive Recording of Children's Learning Environments (CIRCLE) (Version 2.0) is a classroom observation system that provides information to teachers on what adjustments are needed by assessing time-sampled contexts, teacher behavior and child behavior, and using these data to estimate response dependencies (i.e., the probability that a certain child behavior co-occurs with a specific classroom context and teacher behavior). To provide this information, the measure must be sensitive to variations in children's response dependencies, psychometrically sound, and provide benchmarks for decision making. This project aims to strengthen the evidence for the validity of the CIRCLE for use within inclusive preschool settings that use a tiered approach to learning, with the ultimate goal of improving outcomes for young children with or at risk for disabilities.

Project Activities: To validate the CIRCLE, the research team will combine extant data with prospective data collection from the CIRCLE. The extant data will be derived from three completed studies of preschool classrooms. For the prospective sample, data will be collected from preschool classrooms using CIRCLE and existing criterion measures. The project will analyze these data to examine construct, predictive, and criterion validity; benchmarks; moderation effects; and cross-validation/replicability across samples, as well as develop new procedures for using the Tune-up Checklist, a tool for supporting teacher's use of data from the CIRCLE for intervention decision making.

Products: The products of this project will include findings related to the validity of CIRCLE, benchmarks, and new practice knowledge. Products will also include technical information about the instrument, observer training manuals, tablet software, a tool for supporting teachers' data-based decision making, as well as peer-reviewed publications and presentations.

Structured Abstract

Setting: The prospective research will take place in inclusive preschool classrooms in school and community programs (i.e., Federal/state funded Pre-K, Title 1, Head Start) in Kansas and Missouri that serve low-income, at-risk children and children with Individual Education Plans (IEPs). The extant data will be derived from three previous investigations of preschool classrooms that used data collected with the CIRCLE.

Sample: The extant dataset includes 489 preschool children (ages 3-5) in 106 classrooms and at least one teacher per classroom. The prospective sample will include approximately 360 preschool children in 120 classrooms (three target children per classroom) and at least one teacher per classroom. The sample will include preschool children who are typically developing and from low-income backgrounds, children who have IEPs, and children who are dual-language learners (DLL).

Assessment: The CIRCLE is a fully developed observation system for use in preschool classrooms that uses multiple recurring time samples of learning contexts, teacher behaviors, and child behaviors to estimate child response dependencies. The information is intended to guide preschool teachers in implementing and revising interventions when children are not responsive to their current intervention. Observational data is collected using mobile technology (i.e., tablets). The assessment system is designed so that it can be used by a number of different professionals, including school psychologists, early childhood education specialists, teacher trainers, response-to-intervention teams, and researchers. In practice, CIRCLE data is analyzed to create displays of the information to guide intervention adaptations. The data is then to be used to inform the collaborative creation (e.g., between a coach and teacher) of a Tune-up Checklist, a tool to help plan intervention modifications and reach implementation fidelity.

Research Design: The purpose of this project is to validate CIRCLE, an observational measure that assesses the classroom context and teacher and child behaviors, to determine what changes are needed to support children who are unresponsive to existing interventions. More specifically, the research aims to examine the following: (1) whether children's behaviors are significantly dependent on variations in classroom academic content, activity structures, teacher literacy focus contexts, and teacher behaviors (i.e., construct/predictive validity); (2) the benchmarks (base levels occurrence) of CIRCLE events and associated response dependences; (3) whether children's characteristics (age, gender, IEP and dual-language learner status) moderate children's response dependencies; (4) whether exposure to an instructional intervention moderates differences in children's response dependencies in the extant sample (i.e., treatment validity); (5) whether child response dependencies measured by CIRCLE replicate across two time-displaced samples (i.e., cross-validation/replicability); and (6) the relationship of CIRCLE outcomes to existing measures of language, literacy, and social behavior (i.e., criterion validity). To address these questions, a cross-validation design comprised of two, time-displaced samples will be used. For the prospective data, the team will collect data across all four years of the project, with CIRCLE assessments conducted in the middle of the year, criterion outcome measures in the fall and spring, and sample description measures in the fall. The investigators will use generalized linear mixed models (GLMM) to analyze the data.

Control Condition: Due to the nature of the research, there is no control group.

Key Measures: In addition to the primary measure, CIRCLE, the following measures will be used as criterion for the validity study: Preschool Early Literacy Indicator, Get Ready to Read, Test of Preschool Early Literacy, Pre-IPT–Oral Language Tests (English and Spanish), and Social Skills Improvement System-Rating Scale. Measures will also include surveys and assessments to gather background information on the sample to use as predictors in the model, including a child/family demographics questionnaire, teacher preparation and experience survey, and the Teacher Self-Efficacy Scale.

Data Analytic Strategy: Three-level GLMM will be used to analyze response dependencies based on time-sampled categorical data nested within teachers/classrooms. GLMM will be used to estimate the effects of teacher-level (i.e., attained education, self-efficacy) and child-level (i.e., IEP and/or DLL status, socioeconomic status, literacy, age, gender) predictors on response dependencies. They will also use GLMM to examine criterion test measures as predictors of response dependencies.

Project Website: http://earlyliteracyjgcp.res.ku.edu/circle