Project Activities
FH2T is a game-based application that teaches math concepts through discovery-based puzzles. The research team will conduct an efficacy study using student-level random assignment to compare FH2T to two other technology programs designed to teach math concepts and an active control condition. All students will complete pre-tests and post-tests to measure procedural and conceptual understanding of algebraic concepts.
Structured Abstract
Setting
The study will take place in public schools in urban areas of Texas.
Sample
Approximately 3,000 middle school students drawn from approximately 150 mathematics classrooms from 2 large public school districts will participate in this study. Within one district, approximately 65% of students are economically disadvantaged and 20% are English Learners. Within the other district, approximately 92% of students in the district are economically disadvantaged and 19% of students are English Learners.
FH2T is a game-based application that teaches math concepts through discovery-based puzzles. Math symbols are presented as movable, physical objects, and perceptual learning principles and game design elements are incorporated. FH2T has 14 modules that focus on different math concepts and that build on each other, including addition, multiplication, subtraction, division, order of operations, distributive property, factoring, moving expressions from either side of the equals sign, and solving linear equations.
Research design and methods
In Year 1, the research team will improve the data management and visualization system that will enable large-scale data collection during their efficacy study, conduct pilot studies to ensure fidelity, and train teachers on how to use the interventions that will be evaluated. In Year 2, researchers will use a student-level randomized-controlled design and they will randomly assign students within participating classrooms to one of four conditions: (1) FH2T; (2) Dragon Box 12+; (3) solving problem sets in ASSISTments, a web-based homework system that provides hints and immediate feedback to students as they solve problems; or (4) control condition (solving textbook problems on paper). All students will spend 30 minutes of their homework time, 3 times a week, for 4 weeks participating in the study. Two months later, all students will complete a retention assessment. In Year 3, the research team will analyze the data and disseminate the findings.
Control condition
Researchers are comparing FH2T to two counterfactuals (Dragon Box 12+ and ASSISTments) and an active control condition. Dragon Box 12+ is a popular iPad, game-based application designed to teach algebraic concepts through a discovery puzzle-based approach. The efficacy of ASSISTments for improving students' math achievement was demonstrated in a recent IES-funded study. In the active control condition, students will solve sample textbook problems on paper, which is intended to represent business-as-usual instruction.
Key measures
Key measures include the Contrasting Cases Harvard Algebra Project assessment, the State of Texas Assessments of Academic Readiness (STAAR) test, a researcher-developed test of perceptual sensitivity, the Academic Efficacy subscale of the Patterns of Adaptive Learning Scale, the Student Beliefs about Math Survey, the Student Engagement in Math Questionnaire, process data from the FH2T application (e.g., errors, steps, resets, attempts, mouse clicks), and log data to track fidelity of implementation across all conditions.
Data analytic strategy
The research team will use analysis of covariance, hierarchical linear modeling, growth modeling, structural equation modeling, and data visualization methods to answer their key research questions.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Researchers will produce evidence of the efficacy of the FH2T intervention for middle school students. Researchers will also produce peer-reviewed publications as well as disseminate project findings through their website, social media, practitioner-based journals, magazines, newsletters, videos and demonstrations, and other public outlets.
Project website:
Publications:
ERIC Citations: Publications from this project available in ERIC are here.
Select Publications
Journal articles
Decker-Woodrow, L. E., Mason, C. A., Lee, J.-E., Chan, J. Y.-C., Sales, A., Liu, A., & Tu, S. (2023). The Impacts of Three Educational Technologies on Algebraic Understanding in the Context of COVID-19. AERA Open, 9.
Finster, M., Decker-Woodrow, L., Booker, B., Mason, C. A., Tu, S., & Lee, J. E. (2023). Cost-Effectiveness of Algebraic Technological Applications. Journal of Research on Educational Effectiveness, 17(4), 1031–1055.
Ottmar, E., Lee, J.-E., Vanacore, K., Pradhan, S., Decker-Woodrow, L., & Mason, C. A. (2023). Data from the Efficacy Study of From Here to There! A Dynamic Technology for Improving Algebraic Understanding. Journal of Open Psychology Data, 11(1), 5.
Proceedings
Baral, S. (2022, July). Improving Automated Assessment and Feedback for Student Open-responses in Mathematics. In Proceedings of the 15th International Conference on Educational Data Mining, International Educational Data Mining Society.
Baral, S., Botelho, A. F., Santhanam, A., Gurung, A., Erickson, J., & Heffernan, N. T. (2023, June). Investigating patterns of tone and sentiment in teacher written feedback messages. In International Conference on Artificial Intelligence in Education (pp. 341-346). Cham: Springer Nature Switzerland.
Baral, S., Seetharaman, K., Botelho, A. F., Wang, A., Heineman, G., & Heffernan, N. T. (2022, July). Enhancing auto-scoring of student open responses in the presence of mathematical terms and expressions. In International Conference on Artificial Intelligence in Education (pp. 685-690). Cham: Springer International Publishing.
Erickson, J. A. (2020). Natural language processing for open ended questions in mathematics within intelligent tutoring systems. In Proceedings of the 13th International Conference on Educational Data Mining, EDM (pp. 1-4).
Erickson, J., & Heffernan, N. T. (2023, June). Investigating Patterns of Tone and Sentiment in Teacher Written Feedback Messages. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings (p. 341). Springer Nature.
Prihar, E., Lee, M., Hopman, M., Kalai, A. T., Vempala, S., Wang, A., ... & Heffernan, N. (2023, June). Comparing different approaches to generating mathematics explanations using large language models. In International Conference on Artificial Intelligence in Education (pp. 290-295). Cham: Springer Nature Switzerland.
Prihar, E., Moore, A., & Heffernan, N. (2021, June). Identifying struggling students by comparing online tutor clickstreams. In International Conference on Artificial Intelligence in Education (pp. 290-295). Cham: Springer International Publishing.
Razzaq, R., Ostrow, K. S., & Heffernan, N. T. (2020, June). Effect of immediate feedback on math achievement at the high school level. In International Conference on Artificial Intelligence in Education (pp. 263-267). Cham: Springer International Publishing.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
