Co-Principal Investigators: Albano, Anthony; Missall, Kristen; Purpura, David

Purpose: Early mathematics (EM) skills are strong predictors of both short- and long-term achievement. EM competencies are malleable and develop early, with individual differences in children's performance evident prior to kindergarten. This project will develop and validate an efficient, instructionally relevant assessment system in EM.

Project Activities: Researchers will construct a theoretical framework of early mathematics skills and use that framework to iteratively develop and refine a series of items. They will also construct the computer platform through which the assessment will be delivered. Once the items and platform are developed, researchers will complete a series of validity, usability, and feasibility tests of the assessment system. They will estimate the cost of the assessment in applied settings.

Products: The products of this project will include a reliable and valid EM assessment system and information about the cost of the assessment. Researchers will disseminate findings from this study through conference presentations and peer-reviewed publications.

Structured Abstract

Setting: Project activities will occur in early education classrooms across geographic locations representing the eastern (Pennsylvania), central (Indiana), and western (Washington) United States.

Sample: Samples consist of early childhood education lead teachers in participating sites and children enrolled in their classrooms who (1) speak and understand English according to parent or teacher report, (2) are age-eligible to attend kindergarten in the school year subsequent to participation, and (3) have the necessary physical skills and level of compliance for participation.

Assessment: The Early Mathematics-Individual Growth and Development Indicators (EM-IGDI) is an innovative computerized-adaptive assessment that integrates universal screening and diagnostic assessment. EM-IGDI is intended to increase efficiency and accuracy in (1) identifying preschoolers who may be at risk for mathematical difficulties and (2) understanding student strengths and needs to provide appropriate instruction to move students on a trajectory of learning. Assessment activities focus on numbers and operations, shape and space, measurement, and pre-algebraic thinking.

Research Design and Methods: In Phase 1, researchers will construct an overarching theoretical framework based on reviews of research-based consensus documents and empirical literature. This framework defines the EM domains, which will be operationalized in prototype task models and items to be reviewed with teachers and expert consultants and tested with children. Researchers also develop the computer platform. In Phase 2, researchers will refine items and administer them to children across two assessment sessions with additional feedback from teachers and expert consultants. Results from Phase 2 will inform further refinements to the item banks and task models. In Phase 3, researchers will redesign the items and administer them to a new sample of children across two assessment sessions, with statistical analyses repeated from Phase 2 and additional analyses evaluating dimensionality. In Phase 3, they will create materials to support teachers' score interpretation with expert consultant input and then vet these materials with teacher focus groups to ensure usability. In Phase 4, researchers will focus on validation, administering calibrated items within the newly developed computerized-adaptive testing system, and collecting external criterion measures. They will also carry out teacher focus groups to evaluate the utility and feasibility of the final EM-IGDI.

Key Measures: Child measures include the EM-IGDI and the Early Mathematics Diagnostic Assessment. Researchers will use other project-developed measures to assess child engagement, teacher acceptability, and teacher judgment.

Data Analytic Strategy: The research team will conduct quantitative analyses to evaluate item quality, reliability, dimensionality, and criterion-related validity, based on Rasch modeling. Items, task models, and forms will serve as the units of analyses. They will also conduct quantitative and descriptive analyses to evaluate the use (e.g., acceptability, instructional utility) of the EM-IGDI in applied setting.

Cost Analysis: They will estimate the cost of the assessment in applied settings in collaboration with their developers with specific efforts to ensure a reasonable cost of the tool for the end user. The project team will also draw on the expertise of our IGDI consultants as they have extensive experience in marketing and dissemination of similar tools to a similar target audience. They will estimate costs separately for training, assessment materials, and other implementation support (e.g., staffing).