Co-Principal Investigators: Dumas, Denis; Sarama, Julie

Purpose: Math is a core component of cognition and the foundation of academic success, including STEM achievement. Unfortunately, most young children and their teachers do not have access to research-based early childhood math resources. The pandemic further limited such access, especially for underserved children. Building on previous successful research projects, the researchers will develop and evaluate an innovative, integrated, intelligent, and interactive system of technologies based on empirically-validated learning trajectories that will provide the best of personal and digital tools for assessing and supporting children's learning.

Project Activities: Building on previous IES-funded research projects, the research team will add two innovative technologies to the existing website, Learning and Teaching with Learning Trajectories ([LT]2): (1) The Path will be an intelligent leveling tool that guides users through a curriculum path that integrates all 20 LTs. (2) The App will be a user-facing front end providing instruction along that path, housing suites of digital games at the heart of both assessment and instruction and presenting targeted personal (face-to-face) activities from [LT]2. Following the Curriculum Research Framework, development will consist of cycles of formative evaluation from focus groups, surveys, interviews, and system data (e.g., usage) for adults, as well as interviews and system data for children from 3 to 5 years of age, the last featuring new statistical techniques analyzing children's learning growth curves. Beyond informing the development itself, these multiple methodologies also will allow the researchers to evaluate the promise of the intervention on improving both adults' and children's engagement in and learning of early math.

Products: The Bajillions system will integrate empirically validated learning trajectories, learning engineering, and formative assessment to create adaptive games. Final products will include a fully developed integrated digital platform for teaching and learning that will address inequities in early math. The team will develop the intervention and formatively evaluate the extent to which the integrated system increases engagement and learning for both children and adults and identify who benefits most. The project will make a substantive contribution to the advancement of knowledge in three domains: cognition, curriculum, and the scale-up of educational innovations through technology. The formative assessment component will provide teachers with a much-needed tool without requiring them to do additional work.

Structured Abstract

Setting: The research will take place in preschool programs in Colorado and other geographic locations.

Sample: Participants will be selected from three sources: school districts contacted by the Colorado Department of Education, the 30,000+ users of [Learning Trajectories-LT]2 and the development partner, FableVision's list of 45,000 educators. The team will recruit participants from those who work with children aged 3 to 5 years and are willing to participate. Participants will include children and families from racially, ethnically, culturally, and linguistically diverse backgrounds and contexts (families, traditional schools, childcare centers, and in family childcare). Subsamples of adults will range from 8 to 24 teachers and parents in focus groups and interviews to surveys of 100 initially, extending to thousands for later surveys and the system's user data. For children, interviews will be conducted with 12 children at each age, 3 to 5 years, for each digital game, and system data of tens of thousands of children in the final year.

Intervention: The intervention will consist of three integrated technologies: (1) The existing (updated) Learning and Teaching with Learning Trajectories ([LT]2) tool, (2) the system's Path that uses formative assessments from digital games to place children on the integrated-LT path and identifies corresponding instructional activities, and (3) the App front end through which children play games and adults see children's progress and receive personalized activity recommendations. This system will provide the "best of both worlds" by synthesizing approaches or platforms often considered incompatible:

• Learning Trajectories + Just-in-Time Support. The system will encourage, guide, and support everyone to use the knowledge embedded in learning trajectories to support children's mathematical learning and development, providing guidance as needed.
• High + Low Tech (Digital + Personal). Effective learning engages children at their developmental level for each math topic using formative assessment. Both the assessment and adaptive instruction benefit from a combination of high- and low-tech approaches.
• Play and Intentional Instruction–Home and School. Bajillions will incorporate everyday play and routines, playful, intentional activities, and digital playgrounds.
• AI + LTs. Bajillions' recursive adaptions generated through game tasks follow empirically validated research.
• Curriculum + PD. Bajillions retains and enhances [LT]2's effectiveness as a professional development tool, teaching children's development through reports and high-quality Path activities. All components are asset-based and support equity by meeting children where they are and continuously offering children new challenges.

Research Design and Methods: The team will use multiple methodologies to assess the impact for both adults' and children's engagement in and learning of early math using five of the ten formative assessments (piloting) phases of the IES-funded comprehensive Curriculum Research Framework. For example, reactions to mock-ups of the App and Path in focus groups in Year 1 will ensure later adult engagement and learning. In Year 2, researchers will use a survey, follow-up individual interviews, and system data to measure adult engagement and learning. Finally, in Year 3, the project team will use surveys and system data to measure engagement and learning for adults and children.

Control Condition: Contrasts will be between groups and between project data from the proposed study and data from previous IES-funded projects.

Key Measures: Surveys and interviews will be adaptations of those used in the IES-funded TRIAD studies. Critical system data includes usage (frequency, duration, etc.) and for children, data on their level of thinking for each topic and performance on individual tasks within a level, producing rich diagnostic data.

Data Analytic Strategy: For child-achievement data, the researchers will go beyond typical psychometric practice (single observations) that underestimate students' capacity to learn in the future and perpetuate social inequities. They will model formative assessment data to understand and support child learning growth curves. Dynamic Measurement Modeling (DMM) models the nonlinear trajectory by which each individual approaches mastery and the time-point at which that mastery level is reached or is predicted to be reached based on the shape of their growth curve. DMM analyzes both intra- and inter-individual child differences across time, so they will know which games and game characteristics lead to steeper growth curves for whom.

Cost Analysis: The team will use the ingredients method to calculate the costs for all components of implementation of the intervention. Depending on the context (e.g., states, school district, childcare, family,) costs may include staff time, instructional materials, technology, professional development, contracted services, and facility space. For each essential element of the intervention, the research team will provide a detailed accounting of the type and amount of each of the ingredients involved. The team will consider impact measures and cost metrics (child and teacher outcome measures). Once the outcome/impact and costs data have been matched, the study team will develop cost metrics, such as a cost-effectiveness ratio, to provide information about overall program costs, costs per participant and cost per unit of impact. The study team will also conduct a sensitivity analysis to examine how variable the results are to changes in key cost items, such as staff salaries.

Related IES Projects: Comprehensive Postdoctoral Training in Scientific Education Research (R305A070491); Scaling Up TRIAD: Teaching Early Mathematics for Understanding with Trajectories and Technologies (R305K050157); Longitudinal Study of a Successful Scaling-Up Project: Extending TRIAD (R305A120813); Evaluating the Efficacy of Learning Trajectories in Early Mathematics (R305A150243)