|Title:||The Effects of PACT on Social Studies Knowledge and Comprehension Among 8th Grade Students|
|Principal Investigator:||Artzi, Lauren||Awardee:||American Institutes for Research (AIR)|
|Program:||Research Grants Focused on Systematic Replication [Program Details]|
|Award Period:||5 years (07/01/2020 - 06/30/2025)||Award Amount:||$3,999,942|
|Type:||Replication Effectiveness||Award Number:||R305R200002|
Co-Principal Investigators: Hester, Candace; Swanson, Elizabeth; Burns, Matthew; Drummond, Kathryn
Purpose: The purpose of this project is to test the effectiveness of a variation of the Promoting Accelerated Reading Comprehension of Text (PACT) intervention. Previous research on PACT (including this WWC-reviewed study) has shown positive effects for specific social studies content knowledge and more generalized social studies reading comprehension among eighth graders. To build on the extant evidence and to address the need to build adolescents' comprehension skills in content courses, including those from diverse and high need backgrounds, this replication will focus on PACT-Local (PACT-L). Rather than have the PACT developers provide direct support to teachers, PACT-L will use a train-the-trainers (TTT) approach that relies on local coaches to train and coach classroom teachers to implement the intervention.
Project Activities: The team will use a randomized controlled trial (RCT) to test the impact of the PACT-L intervention. The study team will implement the random assignment with different cohorts of schools participating over 3 years. In addition to examining the impact of the intervention on student achievement outcomes, they will measure student motivation and engagement, examine questions of implementation fidelity and sustainability, and assess variability in impact across subgroups.
Products: The researchers will produce information about the effectiveness of the PACT-L intervention for improving student outcomes, as well as information about the costs and cost-effectiveness of the intervention. They will also prepare a released final data set, peer-reviewed publications, and materials to share their findings with education stakeholders.
Setting: This project takes place in diverse districts including Boston Public Schools, District of Columbia Public Schools, and small districts in Missouri and Michigan. Approximately, 150 schools will participate over 3 years.
Sample: The final sample will include 225 social studies teachers and 20,250 eighth-grade students.
Intervention: PACT integrates literacy instruction into American history content. Intervention materials include teacher lesson plans, student reading materials, videos, activity sheets, and assessments. Local coaches will support teachers' implementation of PACT through 2 days of training and 4 coaching sessions involving observations, modeling, and feedback provided to teachers. In the PACT-L TTT model, PACT developers will first train local coaches at an institute and provide ongoing virtual support.
Research Design and Methods: The study uses a cluster RCT in which schools are randomized to treatment and control groups. Schools will be blocked by average English Language Arts (ELA) standardized test scores using the 3 most recent years of scores prior to random assignment. In addition to examining content knowledge and comprehension outcomes, the study team will (1) measure student motivational and engagement outcomes in order to provide more information on the mechanism through which the intervention works, (2) study fidelity of the PACT-L TTT model, (3) investigate the degree to which PACT practices are sustained by teachers 1 year after training and coaching, and (4) examine impacts for high-need student subgroups.
Control Condition: Schools in the control condition will use typical, business-as-usual curriculum and professional development.
Key Measures: Student outcome measures include Gates-MacGinitie Reading Test (GMRT)– Fourth Edition, Test of Silent Contextual Reading Fluency (TOSCRF-2); PACT's Assessment of Social Studies Knowledge and Comprehension; the Modified Assessment of Social Studies Comprehension, which assesses content reading comprehension; and ELA state standardized tests. Researchers will also administer the Learning with Others survey, a student perception survey that measures engagement, intrinsic motivation, and self-efficacy.
Data Analytic Strategy: Researchers will use multilevel regression models, with students nested within teachers and schools, to identify intent to treat impact estimates. Researchers will use additional treatment-by-covariate interaction terms to identify any differential impacts for specific student groups. They will also use observations, logs, classroom recordings, interviews, and surveys to gather data on implementation and on the perceived facilitators of and challenges to implementing PACT-L.
Cost Analysis/Effectiveness: Researchers will use the resource cost model to assess the costs of the PACT-L intervention and its cost-effectiveness in terms of potential improvements to eighth-grade students' outcomes.
Related Projects: The Scale Up of Promoting Adolescents Comprehension of Text (R305A160026); Understanding Malleable Cognitive Processes and Integrated Comprehension Interventions for Grades 7–12 (R305F100013)