Co-Principal Investigators: Johannes, Kristen; Kao, Yvonne; McNeil, Nicole

Purpose: This project will create ICUE Online, an online version of the Improving Children's Understanding of Equivalence (ICUE) intervention, which was developed and tested for efficacy with prior IES funding. ICUE Online will consist of activities with just-in-time feedback designed to improve student understanding of mathematical equivalence. Though understanding equivalence is a critical precursor to many math concepts and algebraic thinking, the majority of elementary students fail to develop correct conceptions and traditional instruction may perpetuate misconceptions. Despite the success of the original ICUE intervention in improving overall rates of proficiency, many students still fail to reach mastery. The original ICUE intervention activities are paper-based workbooks, so students fail to receive immediate feedback on incorrect responses, which may reinforce misconceptions. In this project, researchers will test research-based hypotheses to determine the most effective forms of feedback to incorporate into ICUE Online.

Project Activities: In Year 1, the research team will prototype and develop online versions of the activities and conduct usability studies with approximately 50 students. In Year 2, the research team will conduct feasibility studies and run A/B tests to determine which types and timing of feedback best support learning involving 26 teachers and approximately 650 students. In Year 3, the research team will conduct a pilot study of ICUE Online. In Year 4, the research team will broadly disseminate findings and build awareness of the availability of the freely available ICUE Online materials.

Products: ICUE Online will be a suite of 32 fully developed online activities to support students in developing deep understanding of mathematical equivalence. Activities will be made available through the widely used ASSISTments system, to facilitate uptake and dissemination. In addition, the research team plans to disseminate findings to a broad range of audiences including practitioners, policymakers, and researchers.

Structured Abstract

Setting: The studies will take place in diverse second grade classrooms in urban and suburban California and Indiana schools.

Sample: Approximately 50 second grade students will participate in usability studies, 26 teachers and 600 second grade students will participate in feasibility studies and A/B testing, and approximately 20 teachers and 500 second grade students will participate in the pilot study.

Intervention: ICUE Online will consist of interactive, online versions of 32 activities targeting mathematical equivalence. Students will experience novel types of practice and immediate feedback and teachers will have access to reports that detail common errors and individual student performance. The ICUE Online activities will be deployed using the ASSISTments platform.

Research Design and Methods: In Year 1, the research team will prototype and develop online versions of the activities and conduct usability studies. In Year 2, the research team will conduct feasibility studies and run A/B tests to determine which types and timing of feedback best support learning. In Year 3, the research team will conduct a pilot study of ICUE Online using a randomized controlled trial  design. Teachers and their classrooms will be randomly assigned to either use the new activities or an active control consisting of existing second grade math activities in the ASSISTments system.

Control Condition: For the pilot study, the control condition is an active control consisting of existing second grade mathematics activities available in the ASSISTments system.

Key Measures: Measures to assess usability and feasibility will include verbal protocols from lab studies, classroom observation protocols, teacher interviews, student responses on assessment items, and log data on student actions in the ASSISTments platform. Measures to assess evidence of promise include proximal measures of mathematical equivalence understanding and distal measures such as Mathematics Assessment Resource Service performance tasks.

Data Analytic Strategy: The research team will use a range of data analytic techniques including qualitative coding and analyses, log analyses, descriptive statistics, and hierarchical linear modeling.

Cost Analysis: After the pilot study, the team will carry out a cost analysis of ICUE Online and the existing intervention. Researchers will use findings from the pilot study and cost analyses to prepare a report on the cost effectiveness of the intervention.

Related IES Projects: Arithmetic Practice that Promotes Conceptual Understanding and Computational Fluency (R305B070297), Improving Children's Understanding of Mathematical Equivalence (R305A110198), Improving Children's Understanding of Mathematical Equivalence: An Efficacy Study (R305A150088)