Project Activities
Lens on Science will be based on previous research by the University of Miami team that resulted in a flip-book style preschool science test, which demonstrated preliminary evidence of internal and external validity. The team will expand the current item bank and translate existing and new items to a computer platform. Lens on Science will undergo multiple phases of testing following the guidelines for measurement development from the Standards for Educational and Psychological Testing (AERA, APA, & NCME 1999). Multiple forms of validity evidence will be collected, such as response process evaluation, evaluation of estimation procedures, reliability analyses, and differential item functioning. While data will be collected with children served by Miami-Dade Head Start, the final version of Lens on Science is intended to be used nationally.
Structured Abstract
Setting
The study sample will include Head Start children residing in Miami-Dade County. This county includes a large urban city, surrounding incorporated suburbs, and southern rural farm areas. The Miami-Dade County Head Start program serves over 7,000 low-income children and families. For the past 5 years, science has been a targeted focus in selected Head Start centers in the county. However, given the size of the program and staff turnover, there is currently a wide range of expertise in science from novice to expert in the teacher workforce, and similar variability in the science abilities of the children who will participate in this project.
Sample
Approximately 920 preschool-aged children residing in Miami-Dade County and attending Head Start programs will participate across the 4-year time period of the award.
Intervention
Lens on Science will be an IRT-based computer-administered adaptive test of preschool children's content and processing skills knowledge in science. The test will cover three broad early childhood science content areas (life science, earth/space science and physical/energy science) and is also intended to assess preschool children's science processing skills (observing, describing, comparing, questioning, predicting, experimenting, reflecting, and cooperating).
Research design and methods
Lens on Science will be designed using the guidelines for measurement development from the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999). These guidelines include review of items by a panel of experts for content validation, collecting response-process validation data, collecting data in a non-adaptive framework to obtain estimates of the item parameters, evaluating the estimation procedures, and testing the fully adaptive computerized assessment system.
Control condition
There is no control condition.
Key measures
The research team will use data from the Galileo System for the Electronic Measurement of Learning, the Preschool Learning Behaviors Scale, the Devereux Early Childhood Assessment, and the Learning Express to examine the validity of the preschool science measure.
Data analytic strategy
The measurement framework used in the adaptive assessment will be the Rasch model for dichotomously scored items with Bayesian ability estimation and item selection algorithms. Administration and reporting of the test requires several statistical estimation procedures to compute parameters such as ability levels (using the Bayesian expected a-posteriori estimator), the standard error of the ability estimator (estimated using the standard deviation of the posterior distribution of ability), and mathematical algorithms for item selection (determined by the posterior-weighted item information function).
People and institutions involved
IES program contact(s)
Products and publications
Products: The product of this project will be Lens on Science, a computer-administered, adaptive, IRT-based science summative assessment for preschool children with accompanying administration and technical manuals. The measure is intended to be used to assess children's knowledge of three science content areas (life science, earth/space science and physical/energy science), and children's science processing skills.
Book chapter
Greenfield, D. B. (2015). Assessment in Early Childhood Science Education. In Research In Early Childhood Science Education (pp. 353-380). Springer Netherlands.
Journal article, monograph, or newsletter
Bustamante, A. S., White, L. J., and Greenfield, D. B. (2017). Approaches to Learning and School Readiness in Head Start: Applications to Preschool Science. Learning and Individual Differences, 56: 112-118.
Hu, B. Y., Zhou, Y., Chen, L., Fan, X., and Winsler, A. (2017). Preschool Expenditures And Chinese Children's Academic Performance: The Mediating Effect Of Teacher-Child Interaction Quality. Early Childhood Research Quarterly, 41, 37-49.
White, L. J., and Greenfield, D. B. (2017). Executive Functioning in Spanish- and English-Speaking Head Start Preschoolers. Developmental Science, 20(1).
White, L. J., Alexander, A., and Greenfield, D. B. (2017). The Relationship Between Executive Functioning and Language: Examining Vocabulary, Syntax, and Language Learning in Preschoolers Attending Head Start. Journal of Experimental Child Psychology, 164, 16-31.
Zucker, T. A., Williams, J. M., Bell, E. R., Assel, M. A., Landry, S. H., Monsegue-Bailey, P., ... and Bhavsar, V. (2016). Validation of a Brief, Screening Measure of Low-Income Pre-Kindergarteners' Science and Engineering Knowledge. Early Childhood Research Quarterly, 36, 345-357.
Preparing to Succeed: An Efficacy Trial of Two Early Childhood Curricula
Related projects
Supplemental information
Co-Principal Investigator: Randall Penfield
Purpose: The purpose of this study is to develop an IRT-based, computer adaptive test of preschoolers' science knowledge, Lens on Science, and to gather preliminary evidence of internal and external validity.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
