Improving the Power of Education Experiments with Auxiliary Data

Products: The main product of this research will be flexible, user-friendly, open-source software available to and readily usable by applied education researchers.

Publications:

Botelho, A. F., Prihar, E., & Heffernan, N. T. (2022, July). Deep Learning or Deep Ignorance? Comparing Untrained Recurrent Models in Educational Contexts. In International Conference on Artificial Intelligence in Education (pp. 281-293). Cham: Springer International Publishing.

Cheng, L., Croteau, E., Baral, S., Heffernan, C., & Heffernan, N. (2024). Facilitating Student Learning With a Chatbot in an Online Math Learning Platform. Journal of Educational Computing Research, 07356331241226592.

Gagnon-Bartsch, J. A., Sales, A. C., Wu, E., Botelho, A. F., Erickson, J. A., Miratrix, L. W., & Heffernan, N. T. (2023). Precise unbiased estimation in randomized experiments using auxiliary observational data. Journal of Causal Inference, 11(1), 20220011.

Gurung, A., & Heffernan, N. T. (2022, July). Exploring Fairness in Automated Grading and Feedback Generation of Open-Response Math Problems. In International Conference on Artificial Intelligence in Education (pp. 71-76). Cham: Springer International Publishing.

Gurung, A., Baral, S., Vanacore, K. P., McReynolds, A. A., Kreisberg, H., Botelho, A. F., ... & Heffernan, N. T. (2023, March). Identification, Exploration, and Remediation: Can Teachers Predict Common Wrong Answers?. In LAK23: 13th International Learning Analytics and Knowledge Conference (pp. 399-410).

Gurung, A., Botelho, A., Thompson, R., Sales, A., Baral, S., & Heffernan, N. (2022, November). Considerate, unfair, or just fatigued? examining factors that impact teacher. In Proceedings of the 30th International Conference on Computers in Education. Asia-Pacific Society for Computers in Education.

Gurung, A., Baral, S., Lee, M. P., Sales, A. C., Haim, A., Vanacore, K. P., ... & Heffernan, N. T. (2023, July). How Common are Common Wrong Answers? Crowdsourcing Remediation at Scale. In Proceedings of the Tenth ACM Conference on Learning@ Scale (pp. 70-80).

Gurung, A., Vanacore, K., Mcreynolds, A. A., Ostrow, K. S., Worden, E., Sales, A. C., & Heffernan, N. T. (2024, March). Multiple Choice vs. Fill-In Problems: The Trade-off Between Scalability and Learning. In Proceedings of the 14th Learning Analytics and Knowledge Conference (pp. 507-517).

Haim, A., & Heffernan, N. T. (2022). Student Perception on the Effectiveness of On-Demand Assistance in Online Learning Platforms. In A. Mitrovic and N. Bosch(Eds.), Proceedings of the 15th International Conference on Educational Data Mining, (pp. 734-37), International Educational Data Mining Society.

Haim, A., Baxter, C., Gyurcsan, R., Shaw, S. T., & Heffernan, N. T. (2023, July). How to Open Science: Analyzing the Open Science Statement Compliance of the Learning@ Scale Conference. In Proceedings of the Tenth ACM Conference on Learning@ Scale (pp. 174-182).

Haim, A., Gyurcsan, R., Baxter, C., Shaw, S. T., & Heffernan, N. T. (2023). How to Open Science: Debugging Reproducibility within the Educational Data Mining Conference. International Educational Data Mining Society.

Haim, A., Prihar, E., & Heffernan, N. T. (2022, July). Toward improving effectiveness of Crowdsourced, on-demand assistance from educators in online learning platforms. In International Conference on Artificial Intelligence in Education (pp. 29-34). Cham: Springer International Publishing.

Haim, A., Shaw, S., & Heffernan, N. (2023, March). How to open science: A principle and reproducibility review of the learning analytics and knowledge conference. In LAK23: 13th International Learning Analytics and Knowledge Conference (pp. 156-164).

Haim, A., Shaw, S. T., & Heffernan, N. T. (2023, July). How to Open Science: Promoting Principles and Reproducibility Practices within the Learning@ Scale Community. In Proceedings of the Tenth ACM Conference on Learning@ Scale (pp. 248-250).

Kalai, A. T., Vempala, S., Wang, A., Wickline, G., Murray, A., & Heffernan, N. (2023, June). Comparing Different Approaches to Generating Mathematics Explanations Using Large Language Models. In Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky: 24th International Conference, AIED 2023, Tokyo, Japan, July 3-7, 2023, Proceedings (p. 290). Springer Nature.

Lee, M. P., Croteau, E., Gurung, A., Botelho, A. F., & Heffernan, N. T. (2023). Knowledge Tracing over Time: A Longitudinal Analysis. International Educational Data Mining Society.

Lu, X., Sales, A., & Heffernan, N. T. (2021). Immediate Versus Delayed Feedback on Learning: Do People's Instincts Really Conflict With Reality?. Journal of Higher Education Theory and Practice, 21(16).

Lu, X., Wang, W., Motz, B. A., Ye, W., & Heffernan, N. T. (2023). Immediate text-based feedback timing on foreign language online assignments: How immediate should immediate feedback be?. Computers and education open, 5, 100148.

Mann, C. Z., Sales, A. C., & Gagnon-Bartsch, J. A. (2023). Combining observational and experimental data for causal inference considering data privacy. arXiv preprint arXiv:2308.02974.

Prihar, E., Haim, A., Sales, A., & Heffernan, N. (2022, June). Automatic interpretable personalized learning. In Proceedings of the Ninth ACM Conference on Learning@ Scale (pp. 1-11).

Prihar, E., Lee, M., Hopman, M., Kalai, A. T., Vempala, S., Wang, A., ... & Heffernan, N. (2023, June). Comparing different approaches to generating mathematics explanations using large language models. In International Conference on Artificial Intelligence in Education (pp. 290-295). Cham: Springer Nature Switzerland.

Prihar, E., Moore, A., & Heffernan, N. (2022). Identifying Explanations Within Student-Tutor Chat Logs. In Proceedings of the 15th International Conference on Educational Data Mining (p. 773).

Prihar, E., Sales, A., & Heffernan, N. (2023, June). A Bandit You Can Trust. In Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization (pp. 106-115).

Prihar, E., Syed, M., Ostrow, K., Shaw, S., Sales, A., & Heffernan, N. (2022, July). Exploring common trends in online educational experiments. In Proceedings of the 15th International Conference on Educational Data Mining.

Prihar, E., Vanacore, K., Sales, A., & Heffernan, N. (2023). Effective Evaluation of Online Learning Interventions with Surrogate Measures. International Educational Data Mining Society.

Rodrigo, M. M. T., Vassileva, J., Lane, H. C., Brusilovsky, P., Sosnovsky, S., Biswas, G., ... & Dmitrova, V. (2023). The great challenges and opportunities of the next 20 years. Handbook of Artificial Intelligence in Education, 606-649.

Sales, A. C., Prihar, E. B., Gagnon-Bartsch, J. A., & Heffernan, N. T. (2023). Using Auxiliary Data to Boost Precision in the Analysis of A/B Tests on an Online Educational Platform: New Data and New Results. Journal of Educational Data Mining, 15(2), 53-85.

Sales, A. C., Prihar, E., Gagnon-Bartsch, J., Gurung, A., & Heffernan, N. T. (2022, July). More powerful a/b testing using auxiliary data and deep learning. In International Conference on Artificial Intelligence in Education (pp. 524-527). Cham: Springer International Publishing.

Vanacore, K., Gurung, A., Mcreynolds, A., Liu, A., Shaw, S., & Heffernan, N. (2023, March). Impact of Non-Cognitive Interventions on Student Learning Behaviors and Outcomes: An analysis of seven large-scale experimental inventions. In LAK23: 13th International Learning Analytics and Knowledge Conference (pp. 165-174).

Vanacore, K., Gurung, A., Sales, A., & Heffernan, N. T. (2024, March). The Effect of Assistance on Gamers: Assessing The Impact of On-Demand Hints & Feedback Availability on Learning for Students Who Game the System. In Proceedings of the 14th Learning Analytics and Knowledge Conference (pp. 462-472).

Zhang, M., Baral, S., Heffernan, N., & Lan, A. (2022). Automatic short math answer grading via in-context meta-learning. arXiv preprint arXiv:2205.15219.