Project Activities
The research team will first develop new activities that will promote sense-making and fluency-building and will incorporate these into the Fractions Tutor. Next, the team will gather initial data from students' use of the Fractions Tutor along with the new activities, which will inform the development of the activity selection method. Finally, the research team will conduct two experiments. The first will examine whether robust learning is enhanced in a tutor that supports multiple activity types or supports multiple activity types sequenced adaptively to the individual. The second will ascertain whether there is value in a self-improving version of the activity selection method that leverages data gained as the tutor interacts with more students.
Structured Abstract
Setting
Research will be carried out at several elementary schools located in an urban area in Pennsylvania. The collaborating school districts are diverse in terms of their demographics and academic achievement.
Sample
Initial data collection will occur with approximately 700 fourth- and fifth-grade students, Experiment 1 will occur with between 75-270 fourth- and fifth-grade students, and Experiment 2 will occur with approximately 800 fourth- and fifth-grade students.
The research will produce a new version of the researchers' existing web-based ITS (the Fractions Tutor) for fourth- and fifth-grade fractions learning that will incorporate a wider range of activity types that support a broader set of learning mechanisms and a new, adaptive selection method. The new tutor will support sense-making processes and fluency-building processes, in addition to induction and refinement processes typically supported by intelligent tutoring systems. The new version of the Fractions Tutor will be made freely available on the Mathtutor website, which was funded by a previous IES grant (Bringing Cognitive Tutors to the Internet: A Website that Helps Middle-School Students Learn Math) to the co-PI.
Research design and methods
The research team will carry out activities designed to answer three research questions: (1) Does an ITS support more robust learning when activities are added to support fundamental learning mechanisms more evenly? (2) Does using a sequential decision-theoretic approach to adaptively sequence activities to the individual improve robust learning outcomes compared to a fixed sequence? and (3) Does using a self-optimizing action selection system, which uses accumulating student interaction data to improve its selection mechanism, lead to higher robust learning gains of students who use the system later on compared to students that use the initial system?
To answer these questions, the researchers will carry out three lines of research. First, they will develop new activity types for the Fractions Tutor. These activity types will be developed so that they promote sense-making and fluency-building processes. Next, they will gather initial data in classrooms using the Fractions Tutor in order to inform the development of an adaptive policy that selects among multiple activity types. Lastly, the research team will carry out two experiments. In Experiment 1, they will compare the standard tutor to two new versions of the ITS to evaluate whether robust learning is enhanced in a tutor that (unlike the standard tutor) supports multiple activity types or supports multiple activity types sequenced adaptively by the individual. This is a between-subjects design, where students will be randomly assigned to one of the three conditions. In Experiment 2, the research team will test the value of a self-improving version of the activity selection policy that leverages data gained as the tutor interacts with more students. Data will be conducted serially by classroom in five “waves.” Each wave will be considered a condition and analyses will look for differences between waves.
Control condition
In Experiment 1, the control condition is the current version of the Fractions Tutor. In Experiment 2, the waves of data collection are compared to each other and there is no control condition.
Key measures
The research team will use researcher-designed pre-tests and immediate and delayed post-tests to assess students' procedural and conceptual knowledge. These tests will include standardized test items gathered from readily available sources (e.g., NAEP, state tests, etc.) and items that closely track the Fractions Tutor problem types. In addition, they will log data of students' interactions with the tutor to see how different tutor versions affect learning behavior.
Data analytic strategy
The initial data collected will be used to inform the development of the activity selection method. Experiment 1 and Experiment 2 pre-test and post-test data will be analyzed using multivariate and simple analyses of variance. Researchers will use pairwise comparisons to determine whether students in each later wave performed better than students in each prior wave. In addition, the research team will analyze the temporal course of student performance and see whether parametric models fit the observed data (e.g., if a linear trend in the data is observed).
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The products of this project will be a fully developed, publicly accessible, web-based ITS for fractions learning that effectively leverages a broad range of activity types to achieve more robust learning. Peer-reviewed publications will also be produced.
Publications:
Journal article, monograph, or newsletter
Aleven, V., McLaren, B.M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., Ringenberg, M., and Koedinger, K.R. (2016). Example-Tracing Tutors: Intelligent Tutor Development for Non-Programmers. International Journal of Artificial Intelligence in Education, 26(1): 224-269.
Thomas, P. S., Castro da Silva, B., Barto, A. G., Giguere, S., Brun, Y., & Brunskill, E. (2019). Preventing undesirable behavior of intelligent machines. Science, 366(6468), 999-1004.
Proceeding
Doroudi, S., Holstein, K., Aleven, V., and Brunskill, E. (2016). Sequence Matters, But How Exactly? A Method for Evaluating Activity Sequences from Data. In 9th International Conference on Educational Data Mining (pp. 70-77). Raleigh, NC: International Educational Data Mining Society (IEDMS).
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.