|Title:||Efficacy of ASSISTments Online Homework Support for Middle School Mathematics Learning: A Replication Study|
|Principal Investigator:||Feng, Mingyu||Awardee:||WestEd|
|Program:||Effective Instruction [Program Details]|
|Award Period:||4 years (09/01/2017 - 08/31/2021)||Award Amount:||$3,298,853|
|Type:||Efficacy and Replication||Award Number:||R305A170641|
Previous Award Number: R305A170137
Co-Principal Investigators: Jeremy Roschelle (SRI), Neil Heffernan (Worcester Polytechnic Institute), Jenifer O. Corn (North Carolina State University), Robert Murphy (SRI)
Purpose: Researchers propose to conduct a replication study of the impact of a fully developed, widely adopted intervention called ASSISTments on student mathematics outcomes. Findings from a previous IES-funded efficacy study, conducted in Maine, indicated this intervention led to beneficial impacts on student learning outcomes. The current study will examine the impacts of this intervention with a more diverse sample, use the State standardized test as the student outcome measure, and rely on mathematics coaches (instead of the intervention developers) to provide professional development and support to teachers. These three differences are important to policymakers and educators, who want to know whether interventions work and whether they contribute to closing achievement gaps, produce outcomes on a state-administered assessment, and can be supported sufficiently via local teacher professional development and local implementation conditions and practices.
Project Activities: This intervention includes 1) a web-based platform that provides support to students as they solve mathematics problems from the district-assigned textbooks and provides detailed student-level and class-level formative assessment data to teachers to help inform adjustments in classroom instruction and pacing and 2) professional development to support teacher use of the reports as a formative assessment tool. Schools will be randomly assigned to either the treatment or control condition and will be asked to participate for 2 full school years. Data on implementation fidelity and contrast between conditions will be collected in both years.
Products: Researchers will produce evidence of the efficacy of ASSISTments for Grade 7 math students, and peer-reviewed publications.
Setting: Participating schools will be located in rural, suburban and urban districts across North Carolina. The sample will include low-performing schools and students from lower-socioeconomic-status backgrounds.
Sample: Study participants include approximately 198 seventh-grade teachers and 10,400 students drawn from 66 public middle/junior high schools in the state of North Carolina.
Intervention: ASSISTments is a web-based platform that provides support to students as they solve mathematics problems from the district-assigned textbooks and provides detailed student-level and class-level formative assessment data to teachers to help inform adjustments in classroom instruction and pacing. ASSISTments also provides teachers with access to pre-organized problem sets called Skill Builders to assign to students for additional practice, immediate feedback to student answers, and guidance to students through hints and explanations as they work.
Research Design and Methods: The study will use a school-level, delayed-treatment, clustered randomized experimental design. Schools will be randomly assigned to either the treatment or control condition and will be asked to participate for 2 full school years (2018-19 and 2019-20). Data on implementation fidelity and contrast between conditions will be collected in both years.
In the treatment schools, teachers and students in Grade 7 will use the ASSISTments to support their homework. Teachers will assign homework online and receive diagnostic reports to facilitate their review of homework and adapt their instruction accordingly. Students will complete their homework on their computers and receive (a) immediate feedback on their answers to textbook problems, (b) hint messages to help solve difficult problems, (c) Skill Builders that track and adjust to their mastery status of knowledge, and (d) automatic reassessment to help improve their retention of previous mastered skills. Teachers will receive professional development to support their use of the reports as a formative assessment tool.
Control Condition: In the control condition, teachers will use existing the instructional practices and technologies (other than ASSISTments). Teachers who do not have a formal homework policy will be asked to assign a minimum of 20 minutes of homework per night for at least 3 nights a week.
Key Measures: Primary measures include the North Carolina End-of-Grade assessment as the primary measure of student math achievement and classroom observations as a measure of teacher homework reviews. Other measures include eligibility for free or reduced-price lunch as a measure of socioeconomic status (SES), school size, rural or non-rural status of school, individualized education plan status, and English proficiency status.
Data Analytic Strategy: The team will use a three-level hierarchical linear regression model to examine mean differences in Grade 7 achievement between students in two conditions, controlling for prior achievement and other covariates. Researchers will also complete moderator analyses to examine the impact of the intervention on the learning of students with low-baseline math achievement, and different race/ethnicity and socioeconomic backgrounds. Finally, they will carry out a mediator analysis to examine the link between student learning outcomes and teachers' use of the ASSISTments reports to inform how they review homework review and target classroom instruction.
Related IES Projects: Using Web-based Cognitive Assessment Systems for Predicting Student Performance on State Exams (R305K030140)