Setting
The study occurred in Forsyth County Schools, a district in the suburbs north of Atlanta, Georgia. Eight of the nine middle schools in the district participated in the study.
Study sample
The sample characteristics differed slightly by outcome measure. For the ELA measure: the intervention group was 48% female while the comparison group was 51% female. The intervention group was 32% sixth-graders, 36% seventh-graders, and 33% eighth-graders while the comparison group was 34% sixth-graders, 34% seventh-graders, and 33% eighth-graders. The intervention group was 76% white, 11% Asian, 2% black, 8% Hispanic, 0.30% Indian, 2% mixed race, and 0.03% Pacific Islander while the comparison group was 70% white, 8% Asian, 3% black, 15% Hispanic, 0.4% Indian, 3% mixed, and 0.1% Pacific Islander. The intervention group and comparison groups were both 7% high needs students. For the math measure: the intervention group was 49% female while the comparison group was 51% female. The intervention group was 31% sixth-graders, 36% seventh-graders, and 33% eighth-graders while the comparison group was 34% sixth-graders, 34% seventh-graders, and 32% eighth-graders. The intervention group was 76% white, 11% Asian, 2% black, 8% Hispanic, 0.20% Indian, 2% mixed race, and 0.03% Pacific Islander while the comparison group was 70% white, 8% Asian, 3% black, 16% Hispanic, 0.4% Indian, 2% mixed, and 0.1% Pacific Islander. The intervention group was 6% high needs while the comparison group was 7% high needs.
Intervention Group
The intervention was a learning management system with recommendation engine. The recommendation engine suggested resources that teachers could provide students based on a student's previous achievement and learning preferences. The goal of the recommendation engine was to help teachers provide more personalized instruction to students to enrich their learning. During year three, teachers received professional development focusing on making their instruction more personalized to student needs. The learning management system with recommendation engine was developed in year three and four of the study and its impact was measured in year four. The recommendation engine was not available until February 2014 (year 4), only two to three months prior to the state-mandated tests (post-tests) in April and May of that year. Many instructors reported that they had not implemented the intervention with their classes because they had been focused on preparing their students for the upcoming tests.
Comparison Group
The comparison condition was identical to the intervention condition (i.e., the teachers had access to the learning management system) except that teachers did not have access to the recommendation engine nor did they have training on its use until after the post-test.
Support for implementation
Teachers were given professional development on giving students more personalized instruction in the year before the learning management system was implemented. Fidelity of program implementation was measured by the Program Evaluation Group at the University of Georgia. The Program Evaluation Group determined whether each school integrated the learning management system, trained the professional development trainers, provided professional development, and supported implementation at the school and the district level.