Name of Partners: San Diego Unified School District (SDUSD); University of California, San Diego (UCSD); San Diego Education Research Alliance (SanDERA)

Purpose: The purpose of this project is to address the problem that weak math achievement among middle school students leaves them unprepared for high school-level math course work, thereby putting them at risk for leaving high school, or graduating from high school unprepared for college or employment. Researchers will follow a continuous improvement process as they implement, adapt, and revise a math instruction intervention with four high-poverty middle schools that use professional learning communities (PLCs) for math teachers, ongoing diagnostic assessment of students' math skills and learning needs, provision of instructional materials and strategies, and continuous improvement cycles aimed at improving the quality and rigor of math instruction for all students. By the end of the project, the partnership will have improved strategies for teaching math, and clear processes for assessing students' instructional needs, addressing those needs through differentiated instruction, and maintaining high quality math instruction. This research builds upon an earlier IES-funded researcher-practitioner partnership project, Academic Trajectories and Policies to Narrow Achievement Gaps in San Diego, which created a set of measures that calculate student on-track probabilities for a variety of outcomes.

Project Activities: The SanDERA partnership will coordinate the activities of UCSD researchers and district policymakers, school leaders, and teachers. The partnership will employ its on-track measures, developed during its earlier Researcher Practitioner Partnership project https://ies.ed.gov/funding/grantsearch/details.asp?ID=1432, to assess whether each student is prepared for high school-level math courses. The partnership will feed the on-track measures into professional learning communities (PLCs) at the school level from the start of the school year. An expert math “resource” teacher will coordinate the activities of the PLCs and work with teachers to improve their instruction overall and to identify and address the instructional needs of individual students. Researchers will work with the resource teacher and school leaders to accomplish high-quality, data-driven instruction at the classroom level.

Products: The partnership will produce a refined approach to middle school math instruction that can be implemented across the district, a series of briefs for teachers and school leaders, and papers for presentation at research conferences. Researchers will also produce peer-reviewed journals articles.

Project Website: http://sandera.ucsd.edu

Structured Abstract

Setting: Research activities will take place in the San Diego Unified School District, a large ethnically and linguistically diverse urban district in California.

Sample: The project team will work with approximately 30 math teachers in four high-needs middle schools where 84 – 100 percent of students qualify for free or reduced-cost lunch. Study participants include 3,150 students. In 2013, the schools' largest racial/ethnic subgroups were Hispanic (74%) and African American (9%); 37% of students were English Learners.

Approach: Making the results of diagnostic math assessments available to teachers through innovative online reporting tools is central to the partnership's approach to data-driven instruction. Researchers will work with the resource teacher and school leaders to help teachers interpret assessment data and use it to inform high-quality math instruction aimed at students' instructional needs. Teachers in participating schools will use student-level diagnostic information to group students within the classroom based on individual learning needs; small-group instruction will regularly occur within the classroom. Professional development materials will help teachers to diagnose patterns in the math errors students often make, and provide strategies for correcting those misunderstandings.

Research Design and Methods: School-level and project-wide PLCs will employ two improvement cycles per year, for each of four years during the project. Within each improvement cycle, teachers will gather and analyze formative assessment data in order to evaluate their current pedagogical strategies. They will do this alone, one-on-one with the math resource teacher, in groups during monthly PLC meetings, and during annual Summer Math Summits.

During each of the first cycles per year, researchers will help teachers understand student progress by sharing individual student scores on formative assessments, and providing descriptive statistics to provide classroom-average portraits as well as comparisons of classroom performance to overall performance within the school, for a given math course. At the end of each year, researchers will use descriptive statistics to convey results from the Smarter Balanced Assessment Consortium (Smarter Balanced) Summative Assessments in math, and the Mathematics Diagnostic Testing Project (MDTP) Readiness Tests. The research team will also use regression equations developed in the partnership's prior IES project to compute on-/off-track indicators for all students in the sample, and provide these measures directly to teachers. Within the ongoing comparison study, researchers will use difference-in-difference models to compare math achievement growth for students at improvement and comparison schools.

Control Condition: Four of the district's 25 middle schools have been selected as comparison sites, with matching based on school characteristics as measured by the School Characteristics Index (a demographic index developed by the California Department of Education), student performance on the California Standards Tests in math, and demographic characteristics obtained from district administrative data.

Key Measures: Researchers will use measures to guide the improvement process including the MDTP Written Response Items produced and used by the California State University and University of California systems, and formative Interim Assessment Block measures provided by the Smarter Balanced. Teachers will also use formative assessment tools identified by the National Council of Teachers of Mathematics, to help them understand how students approach a particular problem and how to provide individualized support. Data on the usability of the approach will be obtained through teacher surveys and interviews, observations of PLCs, and document review. Measures for the ongoing comparison will include course grades for middle school math courses and subsequent course work, district-developed end of unit tests, summative measures from the end-of-year Smarter Balanced assessments, and for older cohorts, performance on California's high school exit exam in grade 10.

Data Analytic Strategy: For the quantitative analysis of growth in math achievement at the four project schools and the four comparison schools, researchers will use regression techniques that allow for clustered standard errors, and control for baseline differences in student achievement.

In the qualitative research component of the project, researchers will conduct annual interviews with teachers, principals, and district administrators and observations of monthly PLC meetings in each school and during the annual math summit. The analytical approach involves triangulation of these multiple sources of data in order to identify within and across site themes. The district will also administer semi-annual teacher surveys to gather data on teachers' experiences with the program. Researchers will correlate teachers' reported use of the various assessments and instructional materials from the semi-annual surveys with interview and observational data.

Related IES Projects: Academic Trajectories and Policies to Narrow Achievement Gaps in San Diego (R305H130059)