|Title:||Initial Efficacy Study of Data Wise|
|Principal Investigator:||Arshan, Nicole||Awardee:||SRI International|
|Program:||Effective Instruction [Program Details]|
|Award Period:||5 years (09/01/2020 - 08/31/2025)||Award Amount:||$3,299,879|
Co-Principal Investigators: Boyce, Jared; Schmidt, Rebecca
Purpose: In this Efficacy project, researchers propose to examine the impact of Data Wise on Grade 5 teacher instructional practice and student test scores. This 2-year intervention targets the collaborative processes and data use within instructional teams. This study will provide evidence of the implementation, impacts, costs, and sustainability of this fully developed and widely used approach to data-driven instruction that has no prior evidence of efficacy.
Project Activities: The researchers will test the efficacy of Data Wise using a school-level, clustered randomized control trial blocked within district to assess Data Wise impact for each year of the intervention and one-year post-intervention. They will collect data from participating schools, staff, and grade 5 students.
Products: The project will produce information about the impact of Data Wise on professional development and student learning outcomes and the cost effectiveness of the intervention relative to business-as-usual. The research team will also produce peer-reviewed publications, reports for practitioners, and a final data set to be shared according to the IES Public Access Policy
Setting: The setting for this study is a large district in Texas and multiple small districts in Pennsylvania. Jointly, these districts span rural, suburban, and urban communities.
Sample: The sample will comprise approximately 50 schools, 50 school leaders, 50 instructional experts, 150 teachers, and 4,500 grade 5 students.
Intervention: Data Wise is a 2-year intervention that targets the collaborative processes and data use within instructional teams. An instructional team is a professional learning community consisting of teachers, a school leader, and instructional expert like a school-based coach, instructional leader, or department chair. This instructional team will use multiple sources of data and address specific instructional practices upon which all teachers can improve. The entire instructional team receives the Data Wise intervention. The intervention has four core components: (1) protected meeting time for instructional teams to collaborate, (2) job-embedded tools that lead teams through structured inquiry cycles using multiple forms of data, (3) intensive on-site professional development that models the structured collaborative inquiry-focused process, and (4) monthly virtual coaching for instructional teams.
Research Design and Methods: This study will use a school-level, clustered randomized experimental design blocked within district to answer research questions about short- and long-term student outcomes, proximal outcomes on instructional teams and teachers, moderators, mediators, implementation fidelity, and cost/cost-effectiveness. Outcome data will be collected in first 2 years of the intervention and 1 year after training has been completed.
Control Condition: The comparison condition consists of business-as-usual classrooms that will not receive Data Wise training.
Key Measures: Outcome measures for this study are (1) state standardized grade 5 mathematics test scores (PSSA and STARR) to assess student achievement, (2) direct observation of mathematics instructional practice (CLASS) to measure instructional practice, (3) instructional team meeting logs, and (4) treatment contrast surveys and interviews. Researchers will also collect measures of cost, cost-effectiveness, and implementation fidelity.
Data Analytic Strategy: Researchers will use linear and hierarchical linear modeling to assess the degree to which Data Wise impacts student, instructional team, and teacher outcomes. In addition, they will examine outcomes for subgroups defined by potential individual- or team-level mediators (e.g., prior mathematics achievement or instructional practices) and team- and classroom-level moderators (e.g., instructional practice) that may uncover underlying mechanisms that lead to observed outcomes.
Cost Study: The study will use the ingredients method to estimate start up and maintenance costs in both intervention and comparison teams, allowing for an estimate of cost-effectiveness.