Project Activities
The research team revised an observer impression scale to assess peer social competence of preschool children, establish the psychometric evidence of reliability and validity, and determine its usability for teachers in early learning programs in the U.S. Social-emotional learning is a primary focus area in many publicly funded preschool programs, with increased emphasis placed on this school readiness domain because of the social isolation from peers that many young children experienced during the COVID-19 pandemic. Using classical measurement theory and an iterative process for instrument development, this project established the psychometric feature and instrument usability of a practitioner-administered observer impression scale assessment of preschool children's peer-related social competence. The Teacher Impression Scale (TIS) is an observer rating scale that teachers can use in their classrooms to document the strengths that many children may possess in peer social competence, locate children in their classes who may be in need of more support for being socially engaged with their peers (i.e., children need Tier 2 supports in a Multi-Tiered Systems of Support-MTSS model), and monitor progress when additional support is provided to those children.
Structured Abstract
Setting
The project studies took place in Head Start, state PreK, and community-based early childhood education programs in North Carolina and southern California. There was a mix of urban, suburban, small town, and rural locations.
Sample
Phase 1 of the study, a content analysis, involved 25 key stakeholders (developmental scientists, leaders focusing on race and ethnicity, teachers, administrators, parents) from a variety of settings. Phase 2 involved 38 early childhood education teachers and 141 children. A second study in phase 2 involved 107 teacher and 402 children. Phase 3 involved 39 teachers/classrooms and 78 children.
The TIS is a teacher-administered observer impression scale assessment of preschool children's peer-related social competence. The current version of the scale consists of 14 items, with each item having a 1 (not descriptive) to 5 (very descriptive) rating. Teachers observe an individual child for 5 minutes during free play and complete the ratings immediately after the observation. The observations occur on three different days. The TIS yields a summary rating of children’s peer social competence. The information may be used for general assessment for all children, screening for children who may need support in establishing positive peer social competence, and progress monitoring.
Research design and methods
In the initial phase of the project, members of four constituent groups (developmental scientists; leaders with expertise in racial, ethnic, and cultural responsiveness; practitioners; and family members) completed a Content Validity Index (CVI) evaluation of the original TIS and also participated in virtual focus groups. In the second phase, an initial pilot testing of the scale and subsequent revision was conducted followed by a larger confirmatory psychometric study. From the latter study, investigators also examined the difference in peer social competence for boys and girls and difference in TIS ratings for children whose race/ethnicity matched or did not match the teacher raters. In the third phase, investigators examined the use of the TIS for screening and progress monitoring use of children for whom the classroom teacher had concerns.
Control condition
Due to the nature of this research, there was no control condition.
Key measures
To examine the concurrent validity of the TIS, teachers will complete the Social Skills Improvement System (SSIS).
Data analytic strategy
For phase 1, the investigators conducted a content validity analysis of the Content Validity Index as well as qualitative analysis of focus group narratives. In phase 2, the investigators collected inter-rater agreement, internal consistency, test-retest reliability, concurrent validity, and factor analyses (both exploratory and confirmatory). In phase 3, investigators used inferential statistics to examine discrimination between “low social” and matched typical peer’s social competence (initially and progress across time).
Cost analysis strategy
The investigators employed an ingredients’ model for estimating costs.
Key outcomes
Key findings from this study are reported in Reska et al., 2025 and Sam et al., 2025.
- The content validity study involving key constituent groups led to revision of items and high ratings for instrument useability.
- Pilot and confirmatory psychometric studies led to further revision of TIS and evidence of strong reliability and validity as well as positive user evaluations.
- Positive evidence was found for TIS use in screening and progress monitoring and positive user evaluations for these functions.
- TIS was a relatively inexpensive observational assessment for screening (approximately $26 for screening and approximately $52 for progress monitoring for one child in 2025 dollars). The cost per child is reduced when the TIS is used for multiple children.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
By the end of the project, a fully developed TIS with strong evidence of reliability, validity, and usability will be available for teachers in early learning classes along with a brief 30-minute introduction to procedures for data collection and use of the data once collected. Current plans are to post the TIS for teacher and program use at no cost.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Reszka, S. S., Sam, A. M., Franco-Jenkins, X., Dunn, C. M., & Odom, S. L. (2025). Content Validity of the Teacher Impression Scale Assessment of Preschool Children’s Peer Social Competence. Early Education and Development, 1-17.
Sam, A. M., Reszka, S. S., Dunn, C. M., Franco-Jenkins, X., Tomaszewski, B., Leggett, T., Waters, V. & Odom, S. L. (2025). Initial psychometric analysis of an observational measure to assess preschool children’s peer social competence. Early Education and Development, 1-15, advanced availability online. https://doi.org/10.1080/10409289.2025.2598592
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
