Description:

Previous Award Number: R305A150430
Previous Awardee: University of Texas Health Science Center at Houston

Co-Principal Investigators: Michael Assel, Maria Carlo, Matt Foster, Jeffrey Williams, and Tricia Zucker

Purpose: The purpose of this project is to add science, technology, engineering and math (STEM) assessments to a recently developed direct child assessment system. The School Readiness Curriculum Based Measurement System (SRCBM) was developed to assess young children's language and literacy skills in Spanish and English. Children's pre-academic skills are associated with their academic achievement in kindergarten and later years of school. Although there is a growing interest in supporting and promoting STEM instruction and learning during the early childhood years, there are few standardized assessments of children's knowledge and understanding of STEM concepts. Early childhood educators need assessment tools that can be used to screen children and monitor their progress over time. The research team will further develop the existing measurement system to include assessments of children's knowledge and understanding of math and science concepts to support learning and instruction in STEM areas during the preschool years. Teachers and researchers will be able to use the expanded SRCBM to assess 3- to 5-year-olds' language, literacy, math, and science skills in both English and Spanish.

Project Activities: Project activities will include the development of test items for math and science content areas, an initial pilot test of the items, an in-depth study of the test items and item-level analyses, and an evaluation of the validity and reliability of the math and science scales. The researchers will also develop assessment materials, including a user manual, training for teachers to use the assessment results to inform instruction, and a technical manual summarizing the development and validation of the math and science scales.

Products: The products of this project are fully developed and validated scales to assess early mathematics and science knowledge for English and Spanish speaking preschoolers. Researchers will also produce peer-reviewed publications.

Structured Abstract

Setting: The study will take place in Head Start programs, public school based prekindergarten programs, and private child care centers from across the state of Texas.

Sample: The study sample will include a socioeconomically and linguistically diverse sample of 3- to 5-year-old children, including children who speak English, Spanish, or both. The Year 1 pilot study will include 100 3- to 5-year-old children, distributed equally across age groups. In Years 2 and 3, the sample will include 3,200 children (1,600 in Year 2 and 1,600 in Year 3). In Year 4, 800 children will participate in the reliability and validity study.

Assessment: The existing School Readiness Curriculum Based Measurement System (SRCBM) was developed to assess children's vocabulary, letter sounds, and phonological awareness in English and Spanish. The researchers will expand the SRCBM by developing English and Spanish subtests in the areas of math and science. The SRCBM mathematics scale will assess four areas of mathematics: numbers and counting, operations and early algebraic reasoning, geometry and spatial sense, and measurement. The SRCBM science scale will assess children's content knowledge and process skills in four content areas: physical sciences, life sciences, earth and space sciences, and engineering and technology.

Research Design and Methods: The researchers will complete the measurement development work in four phases. In Year 1/Phase 1, they will develop tasks, directions, and test items for the two scales in each language. In Year 1/Phase 2, they will conduct a pilot study to examine feasibility of testing procedures, item types, verbal and nonverbal directions, and test materials. In Years 2 and 3/Phase 3 of the project, the researchers will conduct a scaling study to evaluate the test items. At the end of Years 2 and 3, they will conduct item-level analyses to evaluate the new test items and make revisions. In Year 4, the researchers will conduct a study to evaluate and compare the reliability, validity, and sensitivity to growth of the new measures to existing measures of math and science concepts. They will administer a battery of researcher-developed and standardized measures to children, conduct analyses to examine the psychometric properties of the math and science subscales, and finalize the assessment materials. The researchers will produce two progress monitoring kits, one in English and one in Spanish, a training protocol for teachers, a user manual, and a technical manual.

Control Condition: There is no control condition in this study.

Key Measures: The Picture Vocabulary and Vocabulario sobre Dibujos subtests from the Woodcock-Muñoz Language Survey—Revised Normative Update will be used as a screening measure for children's English and Spanish vocabulary skills. Primary measures include researcher-developed and standardized measures of mathematics and science, including the Center for Improving the Readiness of Children for Learning and Education (CIRCLE) Phonological Awareness, Language Literacy System (C-PALLS) math and science subscales, the Individual Growth and Development Indicators-IGDIs Early Numeracy scale, Research-Based Early Math Assessment, and the Preschool Science Assessment/Lens on Science measures.

Data Analytic Strategy: In Years 2 and 3, the research team will use Item Response Theory modeling to identify and retain reliable, discriminating, unbiased items that span the ability continuum for English and Spanish speaking preschoolers. To account for nested data, researchers will use multilevel modeling procedures in SAS and Mplus to examine the validity and reliability of the math and science scales.

Project Website: The new products will be available online via a university website. Educators and researchers interested in using the measurement system will pay a small fee to download and use the measure.

Related IES Projects: Development of the School Readiness Curriculum Based Measurement System (R305A110549), Lens on Science: Development and Validation of a Computer-Administered, Adaptive, IRT-Based Science Assessment for Preschool Children (R305A090502)