Project Activities
The study will test the efficacy of the use of ALEKS by two cohorts of students, each participating for one academic year. Teachers will participate in both years (new teachers will be recruited to participate in Year 2, as necessary). Teachers will receive training on the intervention during the summer immediately before implementation at the start of the fall semester. The research team will develop rubrics for assessing fidelity of implementation to be used during classroom visits. Also, the team will develop surveys for students, conduct interviews with teachers, and collect classroom artifacts (e.g., lesson plans, logs, and teacher-developed assessments). The district will administer the state end-of-course algebra exam.
Structured Abstract
Setting
The study will take place in ten schools located in a large school district located in a major metropolitan city in Pennsylvania.
Sample
Twenty teachers - two per school - will participate, as well as their students from two of each teacher's sections of Algebra 1, for a total of no more than 1,320 students per year. It is anticipated that most of the students will be 9th graders. Student demographic characteristics of the district overall are about 55% African American, 19% Hispanic, 14% white, and 8% Asian; about 82% of students are eligible for free and reduced-price lunch, and about 8% are ELL.
ALEKS is a web-based intelligent tutoring system for Algebra 1. The system assesses individual student knowledge, provides individualized mathematics instruction based on the data gathered from assessments and learning, and reassesses the student periodically in the form of progress assessments to ensure learning and mastery. The system first conducts an initial assessment of the students' ability level with an adaptive assessment, about 20 to 30 questions, that is algorithmically generated. The system then directs the students to topics it determines the student is ready to learn, allowing the student to choose a particular topic for study and for work on problems. Most problems are constructed-response questions, thereby avoiding multiple choice responses and instead requiring the students to demonstrate content mastery. Students receive immediate feedback and step-by-step explanations (e.g., worked examples). Other supports include links to a central dictionary of definitions for mathematical terms and concepts. Students' progress is monitored by the system, and used to establish learning, retention, and mastery. The system will generate reports for the teachers that may inform instructional decisions.
Research design and methods
The team will test the intervention using a randomized control trial, with teachers serving in both the Treatment and Comparison conditions. At each school, two teachers will participate, with each teaching two sections of algebra. One teacher will be randomly assigned to the treatment condition for her earlier section and to the comparison condition for her later one; the other teacher will have the opposite assignments. Assignment of students to sections will occur in two stages. First, the school will initially assign students' math section to be either the earlier or later time period; second, the research team will randomly assign students to either the treatment or comparison section for that time period. The team will develop a set of rubrics for assessing fidelity of implementation, surveys for students, and interview protocol for use with teachers. The team will also collect classroom artifacts to support the assessment of fidelity of implementation.
Control condition
Students assigned to the comparison group will receive the school's standard algebra instruction.
Key measures
The primary outcome measure of student learning will be the score of algebra achievement on the Pennsylvania Keystone Algebra Exam. This exam consists of two modules: operations and linear equations & inequalities, and linear functions and data organization. Student surveys will measure confidence in and attitudes about mathematics and technology, future course taking, schooling and career plans. Additionally, there will be researcher-developed measures of teachers' direction and support of students' use of the software.
Data analytic strategy
Data analyses will include hierarchical linear models for test scores and generalized linear mixed models for dichotomous outcome measures, accounting for possible effects of classroom period, differences between implementation years, and teachers. Possible moderators will be students' background characteristics such as race/ethnicity, language status, poverty, and gender, with covariates including student demographic characteristics and the prior year's score on the Pennsylvania System of School Assessment (PSSA). Also, researchers will examine, through quantitative and qualitative measures, variation in implementation, available resources, and type and quality of assistance provided by principals and teachers.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Products from this study will be evidence of the efficacy of the intervention. The study will also produce peer reviewed publications and briefs accessible to practitioners and policy makers.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
