Project Activities
A multi-site randomized controlled trial will be conducted to determine the efficacy of PACT for eighth-grade students with disabilities in general education social studies classrooms. Two cohorts of eighth-grade general education social studies teachers and their students (from four classes per teacher) will be recruited to participate in the study for two years. Matched pairs of classes within teacher will be randomized to either the PACT intervention or the business-as-usual condition. Teachers will receive professional development and coaching through workshops, booster sessions, and in-class coaching to implement the intervention. The researchers will examine the impact of PACT on the reading outcomes of students with disabilities and the conditional effects of teacher- and student-level factors using mediation and moderation analyses. They will also examine the costs and cost-effectiveness of PACT.
Structured Abstract
Setting
The research will take place in urban and suburban middle schools in Texas and Tennessee.
Sample
Approximately 32 schools and 80 eighth-grade social studies teachers will participate in the study. Students in four social studies classes taught by each teacher in each of the two years will also participate for a total of approximately 16,000 eighth-grade students. Each teacher's class is expected to have two or more students with disabilities; thus the sample will include approximately 1,280 students with disabilities.
Intervention
PACT is a fully developed text- and inquiry-based reading comprehension intervention that was developed and tested for efficacy with secondary students in social studies classrooms as part of a previously-funded IES project. The PACT intervention is a set of instructional supports for teachers and materials for students that is made up of the following components: an overarching question that guides students through a unit, essential word instruction, content acquisition through text reading, and team-based learning activities. The intervention is implemented by teachers daily in general education social studies classrooms for approximately 8-10 consecutive weeks during the fall semester. The research team provides a 1-day professional development wokshop to participating teachers on PACT practices which addresses the following topics: (a) an overview of the study, (b) a detailed description of the experimental design with an emphasis on adherence to intervention procedures, (c) implementation of PACT intervention components, and (d) classroom practices to facilitate discussion focused on text evidence. After the first unit, teachers receive a 1-day follow-up booster session where researchers review components of the intervention, address questions, and problem solve any areas of challenge. In-class coaching is provided during implementation of each of the three units.
Research design and methods
Researchers will conduct a multi-site randomized controlled trial to determine the efficacy of the PACT intervention for students with disabilities in eighth-grade general education social studies classes. In Year 1, eighth-grade general education social studies teachers will be recruited to participate in Cohort 1 of the study with data collected in Years 2 and 3. Additional eighth-grade general education social studies teachers will be recruited to participate in Cohort 2 with data collected in Years 4 and 5. Each cohort will include 20 teachers per state (Texas and Tennessee), and each teacher will teach approximately 4 social studies classes for a total of 80 classes per state per cohort and approximately 2,000 students (including 160 students with disabilities) per state per cohort. Researchers will randomize classes within teacher to either the PACT intervention or business-as-usual condition. First, classes will be matched based on pre-reading scores of students with disabilities and then the matched pairs will be randomized to either the PACT intervention or business-as-usual condition within teacher. The researchers will examine the impact of PACT on the outcomes of students with disabilities and the mediating and moderating effects of teacher- and student-level factors. Researchers will assess implementation fidelity and account for any crossover between the intervention and business-as-usual classes. The researchers will conduct a cost analysis to determine the cost of the intervention over and above what schools typically spend on instruction and an incremental cost-effectiveness analysis.
Control condition
Classes assigned to the business-as-usual condition will engage in the school's usual social studies instruction.
Key measures
Teacher measures include observations of teachers' academic interactions with students with disabilities and fidelity of PACT implementation using researcher-created data collection instruments from prior studies. Measures to assess student learning include a standardized measure of reading comprehension (Gates-MacGinitie Reading Test-4th Edition) and a measure of social studies content knowledge (Assessment of Social Studies Knowledge and Comprehension). Student demographic data will also be collected to assess potential moderators of English language learner status and special education status.The costs of implementing the program will be assessed using the ingredients approach. Data will be collected from the project budget and interviews with project and school personnel to catalogue all resources that would be needed to replicate every aspect of PACT beyond those already required for typical instruction. Costs for all ingredients will be based on national prices provided in the Center for Benefit-Cost Studies in Education's Cost Out Tool Kit.
Data analytic strategy
The researchers will use multilevel multiple-group regression and structural equation models (estimating three-level models with classes at Level 1, teachers at Level 2, and students at Level 3) to estimate the impact of PACT. Moderating effects of student variables and mediating effects of fidelity will be evaluated as cross-level interactions within the multi-level model. For the implementation study, researchers will estimate compliance-adjusted effects using an instrumental variable approach, where "compliance" represents levels of fidelity of implementation. For each key student outcome in the efficacy study, the resulting effect sizes and costs associated with the program will be used to compute incremental cost-effectiveness ratios.
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project will include evidence of the efficacy of the PACT reading comprehension intervention for students with disabilities. The project will result in a final dataset to be shared, peer-reviewed publications and presentations, and additional dissemination products that reach education stakeholders such as practitioners and policymakers.
Project website:
Related projects
Supplemental information
Co-Principal Investigator: Roberts, Greg; Martinez, Leticia; Wanzek, Jeanne
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
