Name of Partners: WestEd, Pasadena Unified School District, Rialto Unified School District, Tehama Department of Education, and Compton Unified School District

In this research project, the partnership will work with elementary schools to improve implementation of ) a What Works Clearinghouse-reviewed curriculum called pre-K Mathematics, that has been evaluated in previous IES funded projects so that it meets the learning needs of low-income students attending these schools. Although the implementation of Pre-K Mathematics has led to significant gains in mathematics skills when tested during randomized trials, researchers and district administrators have noted that implementation of the curriculum often fades after the evaluation is over. The partnership will enhance supports for implementation of Pre-K Mathematics, with the intent of embedding the curriculum within teachers' instructional practice and within the districts' systems of pre-K instruction. Through a “scaling in” process, researchers will preserve and enhance delivery of the essential features of the intervention that are necessary to produce positive math outcomes and modify non-essential features judged unsustainable by the districts. By improving the pre-K math curriculum, the partnership intends to increase student proficiency in math at kindergarten entry, thus reducing the gap in math knowledge across income groups at the beginning of elementary school. At the end of the project, the partnership will have an improved Pre-K Mathematics curriculum tailored to the needs of each district, and each district will have greater capacity for implementing this curriculum.

Project Activities: The partnership will coordinate activities of the research team, district administrators, and teachers. The partnership will rely on input from teachers regarding necessary conditions for implementing the intervention and upon district administrators for supporting and monitoring implementation. In year one, the partnership will implement the standard Pre-K Mathematics curriculum with modifications chosen by district administrators and teachers. Through two short semester-long improvement cycles followed by a single annual cycle the partnership will revise and adapt the Pre-K Mathematics curriculum so that it is tailored to the needs of low-income students as well as the organizational features of the four participating districts. The partnership will communicate its findings to policymakers in California, sharing strategies for improving pre-K instruction across the state.

Products: The products of this research will include a modified Pre-K Mathematics curriculum for each participating district, a report on the policy implications of the study's findings made available through WestEd's website, and peer-reviewed publications.

Structured Abstract

Setting: This project will take place in schools located in four geographically-diverse districts in California.

Sample: Study participants will include two cohorts of 32 teachers, for a total of 64 teachers in 64 classrooms located in four districts. Each of the two cohorts will include a sample of 12 students per classroom students in each of 2 study years. Thus, Cohort 1 and Cohort 2 will each include about 380 students across 32 classrooms, in each of the 2 study years. Researchers will track the Year One sample of about 760 students during the semester-long improvement cycles. For the comparison study, researchers will track the Year Two sample of about 760 students during each cohort's year-long improvement cycle. Over 70 percent of students in these schools receive free or reduced-price lunches; between 17 and 33 percent are English learners; and between 60 and 100 percent come from racial/ethnic minority groups (African American, Asian, Hispanic, and Other).

Approach: The standard form of the Pre-K Mathematics intervention combines professional development for teachers with content aligned with the Common Core State Standards. Through workshops and onsite training, teachers learn pedagogical content knowledge and strategies for delivering the curriculum with fidelity, including the use of small-group activities and engaging parents in home-based math activities. Based on the needs of students and organizational capacity within each of the districts, the partnership will modify the standard curriculum by

• Providing curriculum training to assistant teachers so that they can support delivery of the curriculum;
• Funding floating program assistants to move between pre-K classrooms so that they can maintain order while small-group instruction is taking place;
• Engage teachers in a process of reflection and dialogue regarding how to modify the curriculum to enhance its effectiveness and sustainability; and
• Put in place a set of administrative procedures whereby district administrators will actively monitor and support pre-K Mathematics implementation.

Research Design and Methods: This project combines two short, semester-long improvement cycles during Year one with an annual improvement cycle during Year two (for two cohorts). During the short cycles, researchers will work with teachers to gauge whether they are on track or have fallen significantly below the recommended dosage level for the Pre-K Mathematics curriculum. Trainers will work with teachers who have fallen below recommended levels to assess their practice as well as their needs for instructional support. During the long cycles, classroom observers will assess teachers' instructional practices more broadly, including the conceptual breadth of their instruction and the specific pedagogical techniques they employ to deliver the curriculum. During these cycles, researchers and district staff will convene to review instructional data from improvement and comparison classrooms and assess the need for systemic reforms to support high-quality pre-K instruction. To assess the effectiveness of curriculum modifications, researchers will randomly assign classrooms to improvement and comparison classrooms and will compare students' scores on the ECLS-B assessment during Year Two of the study, for both cohorts.

Control Condition: Thirty-two classrooms, with a total of about 380 students in both years of the study, will be chosen at random to serve as comparison sites. These classrooms will be located in the same districts and, in some cases, the same schools as the improvement classrooms. Students within these classrooms will receive the standard Pre-K Mathematics curriculum as part of an i3Validation study. Researchers will monitor instructional practices in the comparison classrooms using identical teacher surveys and fidelity checklists as those used in the improvement classrooms. Comparisons of student outcomes across the improvement and comparison classrooms will take place over a 2-year period for both cohorts.

Key Measures: Trainers will use the Fidelity of Implementation Record Sheet during bi-weekly classroom observations of small-group math sessions to evaluate teachers' fidelity to the intervention. Trainers and data collectors will also use an adaptation checklist to document implementation of modifications to the standard pre-K Mathematics curriculum by teachers and administrators. Teachers will use the Assessment Record Sheet (ARS) to record each small group math session, including an assessment of each child's performance during the session. Teachers will collect Parent Feedback Forms (PFFs) from parents, to track the extent and perceived usefulness of home math activities encouraged through home math kits. Teachers will transfer ARS and PFF data to a dosage checklist, which will facilitate comparisons to benchmarks, and between improvement and comparison classrooms. Researchers will use the Early Mathematics Classroom Observation tool to measure teachers' instructional practices and pedagogical strategies. The research team will measure short-term student outcomes using a math mastery instrument developed by WestEd. To measure annual achievement growth, assessors will administer the ECLS-B preschool mathematics assessment at the beginning and end of Year two.

Data Analytic Strategy: To compare dosage delivery across improvement and comparison classrooms, researchers will compute chi-square statistics to compare deviations from benchmark dosage levels. Researchers will compare average math scores, as well as classroom variation in math scores, across improvement and comparison classrooms. Researchers will use a generalized linear regression model to assess whether implementation and short-term student outcomes differ significantly between improvement and comparison classrooms and whether apparent differences are moderated by teacher characteristics and/or dosage. Researchers will use a two-level generalized linear regression model of students within classrooms to estimate whether mathematical achievement differs significantly between improvement and comparison classrooms.

Related IES Projects: Scaling Up the Implementation of a Pre-Kindergarten Mathematics Curriculum in Public Preschool Programs (R305K050186), Closing the SES Related Gap in Young Children's Mathematical Knowledge (R305A080188), A Randomized Study of the Efficacy of a Two-Year Mathematics Intervention for At-Risk Pre-Kindergarten and Kindergarten Students (R305A120262); and A Longitudinal Study of the Effects of a Pre-Kindergarten Mathematics Curriculum on Low-Income Children's Mathematical Knowledge (R305J020026)