Description:

Co-Principal Investigator: Randall Penfield

Purpose: Recent emphasis has been placed on science as a school readiness domain. However, an obstacle to conducting research and evaluation on early childhood science programs is the lack of appropriate, reliable, and valid direct assessments of children's science knowledge and process skills. This measurement project will fill the need for a reliable and valid direct assessment of preschool children's science. A collaborative team from the University of Miami, the Miami-Dade County Head Start/Early Head Start program, and Children's Progress will create an Item Response Theory (IRT)-based, computer adaptive test of science, Lens on Science, primarily for use with low-income preschool children.

Project Activities: Lens on Science will be based on previous research by the University of Miami team that resulted in a flip-book style preschool science test, which demonstrated preliminary evidence of internal and external validity. The team will expand the current item bank and translate existing and new items to a computer platform. Lens on Science will undergo multiple phases of testing following the guidelines for measurement development from the Standards for Educational and Psychological Testing (AERA, APA, & NCME 1999). Multiple forms of validity evidence will be collected, such as response process evaluation, evaluation of estimation procedures, reliability analyses, and differential item functioning. While data will be collected with children served by Miami-Dade Head Start, the final version of Lens on Science is intended to be used nationally.

Products: The product of this project will be Lens on Science, a computer-administered, adaptive, IRT-based science summative assessment for preschool children with accompanying administration and technical manuals. The measure is intended to be used to assess children's knowledge of three science content areas (life science, earth/space science and physical/energy science), and children's science processing skills.

Structured Abstract

Purpose: The purpose of this study is to develop an IRT-based, computer adaptive test of preschoolers' science knowledge, Lens on Science, and to gather preliminary evidence of internal and external validity.

Setting: The study sample will include Head Start children residing in Miami-Dade County. This county includes a large urban city, surrounding incorporated suburbs, and southern rural farm areas. The Miami-Dade County Head Start program serves over 7,000 low-income children and families. For the past 5 years, science has been a targeted focus in selected Head Start centers in the county. However, given the size of the program and staff turnover, there is currently a wide range of expertise in science from novice to expert in the teacher workforce, and similar variability in the science abilities of the children who will participate in this project.

Population: Approximately 920 preschool-aged children residing in Miami-Dade County and attending Head Start programs will participate across the 4-year time period of the award.

Intervention: Lens on Science will be an IRT-based computer-administered adaptive test of preschool children's content and processing skills knowledge in science. The test will cover three broad early childhood science content areas (life science, earth/space science and physical/energy science) and is also intended to assess preschool children's science processing skills (observing, describing, comparing, questioning, predicting, experimenting, reflecting, and cooperating).

Research Design and Methods: Lens on Science will be designed using the guidelines for measurement development from the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999). These guidelines include review of items by a panel of experts for content validation, collecting response-process validation data, collecting data in a non-adaptive framework to obtain estimates of the item parameters, evaluating the estimation procedures, and testing the fully adaptive computerized assessment system.

Control Condition: There is no control condition.

Key Measures: The research team will use data from the Galileo System for the Electronic Measurement of Learning, the Preschool Learning Behaviors Scale, the Devereux Early Childhood Assessment, and the Learning Express to examine the validity of the preschool science measure.

Data Analytic Strategy: The measurement framework used in the adaptive assessment will be the Rasch model for dichotomously scored items with Bayesian ability estimation and item selection algorithms. Administration and reporting of the test requires several statistical estimation procedures to compute parameters such as ability levels (using the Bayesian expected a-posteriori estimator), the standard error of the ability estimator (estimated using the standard deviation of the posterior distribution of ability), and mathematical algorithms for item selection (determined by the posterior-weighted item information function).

Related IES Projects: Early Childhood Hands-On Science Curriculum Development and Demonstration (R305K060036), ECHOS: Early Childhood Hands on Science (R305A100275), and Enfoque en Ciencia:Extending the Cultural and Linguistic Validity of a Computer Adaptive Assessment of Science Readiness for Use with Young Latino Children (R305A130612)

Products and Publications

Book chapter

Greenfield, D. B. (2015). Assessment in Early Childhood Science Education. In Research In Early Childhood Science Education (pp. 353–380). Springer Netherlands.

Journal article, monograph, or newsletter

Bustamante, A. S., White, L. J., and Greenfield, D. B. (2017). Approaches to Learning and School Readiness in Head Start: Applications to Preschool Science. Learning and Individual Differences, 56: 112–118.

Hu, B. Y., Zhou, Y., Chen, L., Fan, X., and Winsler, A. (2017). Preschool Expenditures And Chinese Children's Academic Performance: The Mediating Effect Of Teacher-Child Interaction Quality. Early Childhood Research Quarterly, 41, 37–49.

White, L. J., and Greenfield, D. B. (2017). Executive Functioning in Spanish- and English-Speaking Head Start Preschoolers. Developmental Science, 20(1).

White, L. J., Alexander, A., and Greenfield, D. B. (2017). The Relationship Between Executive Functioning and Language: Examining Vocabulary, Syntax, and Language Learning in Preschoolers Attending Head Start. Journal of Experimental Child Psychology, 164, 16–31.

Zucker, T. A., Williams, J. M., Bell, E. R., Assel, M. A., Landry, S. H., Monsegue-Bailey, P., ... and Bhavsar, V. (2016). Validation of a Brief, Screening Measure of Low-Income Pre-Kindergarteners' Science and Engineering Knowledge. Early Childhood Research Quarterly, 36, 345–357.

Preparing to Succeed: An Efficacy Trial of Two Early Childhood Curricula