Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Testing the Efficacy of the CAL-KIBO Educational Robotics Curriculum for Improving Computational Thinking, Fluid Reasoning, and Math Achievement
Center: NCER Year: 2022
Principal Investigator: Willoughby, Michael Awardee: RTI International
Program: Science, Technology, Engineering, and Mathematics (STEM) Education      [Program Details]
Award Period: 5 years (07/01/2022 – 06/30/2027) Award Amount: $3,799,853
Type: Initial Efficacy Award Number: R305A220032

Purpose: In addition to ensuring that students master academic skills, educators are increasingly charged with fostering a range of cognitive competencies that foster student success in the global workplace. The researchers will test whether the CAL-KIBO curriculum, an educational robotics program designed for use with early elementary school-aged students, improves children's computational thinking, fluid reasoning, and math achievement.

Project Activities: The researchers will evaluate the efficacy of CAL-KIBO using an experimental, blocked-randomized design such that classrooms within each grade of each school will be randomly assigned to an active treatment or business as usual (BAU) condition. The researchers will recruit 570 students from 57 first- and second-grade classrooms. The active treatment will involve the delivery of the CAL-KIBO curriculum (18 hours of activities, typically 24, 45-minute sessions) during Specials classes across a 14-week intervention period. Teachers who deliver CAL-KIBO will receive all required materials, participate in two days of professional development related to the curriculum and its pedagogical philosophy, and receive weekly coaching to support their implementation of classroom activities. Classrooms that are assigned to the BAU condition will participate in Specials classroom activities as is customary in each school.

Products: This study will provide the first large-scale, experimental test of the CAL-KIBO classroom curriculum, and our results will be of strong interest to numerous educational stakeholders. The researchers will also produce peer-reviewed publications and a final publicly accessible dataset.

Structured Abstract

Setting: Researchers will carry out the study in first- and second-grade classrooms in central North Carolina.

Sample: Participants will include 570 first- and second-grade children who reflect the racial, ethnic, and socioeconomic diversity of central North Carolina.

Intervention: CAL-KIBO is an educational robotics program with a supporting pedagogical approach that is intended for use in early elementary grades. CAL-KIBO involves 18 hours of activities that will be delivered by Specials teachers during a 14-week intervention period.

Research Design and Methods: A multisite cluster randomized controlled trial will be used to test the efficacy of the CAL-KIBO classroom curriculum against a business-as-usual (BAU) condition. The team will adopt a blocked-randomized research design: all the classrooms in a single grade (first or second) in a single school will constitute a block. Classrooms will be randomly assigned to the CAL-KIBO or BAU condition within each block. Pretest assessments and teacher training will be completed in the fall of the school year. The treatment will be delivered in the winter, and posttest assessments will occur in the spring. Children will complete the same assessment battery at pre- and posttest assessments.

Control Condition: Classrooms that are assigned to the business-as-usual condition will attend their typical Specials classes (for example, art, music).

Key Measures: Outcome measures include student performance-based assessments of computational thinking (TechCheck), fluid reasoning (Weschler Intelligence Scale for Children), and math achievement (Applied Problems of the Woodcock-Johnson Tests of Achievement).

Data Analytic Strategy: The team will use mixed linear models to test for group differences at posttest (main effects), and whether treatment effects differ as a function of child characteristics (moderation effects).

Cost Analysis: The researchers will use a prospective data collection approach, ensuring that costs are measured close to the time incurred and limiting the possibility of recall bias in cost estimates. A bottom-up costing methodology will be applied to identify each resource used or lost and to collect information on the quantity and unit value of each resource. Our primary focus will be on collecting costs from the school perspective. The total CAL-KIBO program costs for all students will be estimated, not just the subset who participate in the study. The researchers will combine these estimates with program output data to estimate costs per classroom and per participating student. Cost data with program output data will be combined to generate estimates of average program startup costs, ongoing program (i.e., non-research) costs per classroom and per student from the perspective of the school or district, and, if any, non-school/district costs from the societal perspective.