Skip Navigation

Examining the association between schoolwide instructional measures and school-level student outcomes in Massachusetts' low performing schools

Northeast & Islands

Description: Since 2010, the Massachusetts Department of Elementary and Secondary Education (ESE) has made a significant investment in developing a coherent and aligned system for supporting and monitoring its low-performing schools. Throughout this monitoring process, ESE has gathered quantitative and qualitative data, including schoolwide instructional observation data, to provide formative feedback to schools on their progress in implementing turnaround practices. ESE also uses these data to target support to individual schools and identify needs across districts and regions and throughout the state annually and over time.

ESE recognizes that the monitoring process needs further examination to ensure that the focus is on high-leverage strategies that demonstrate evidence locally for improving student outcomes. REL Northeast & Islands is working with ESE to conduct analyses of the schoolwide instructional observation data collected using Teachstone's Classroom Assessment Scoring System (CLASS) tool over time at the state's low-performing schools. Researchers will examine whether the schoolwide instructional observation data gathered during monitoring visits are associated with improved schoolwide student achievement and growth. To conduct the analyses, the study will use school-level English language arts (ELA) and math achievement data and the average schoolwide CLASS observation scores in 98 low-performing schools covering elementary, middle and high schools, in the 2014/15, 2015/16, and 2016/17 school years. The information from the study will be used to support ESE and participating schools and districts in interpreting the classroom observation results.

Research Questions:

  • What are the characteristics of Massachusetts' low-performing schools in terms of CLASS observation scores, schoolwide student achievement and growth, and student demographics (race/ethnicity, English learner students, students with disabilities, and students eligible for free and reduced-price lunch)?
  • Are the CLASS observation scores of the low-performing schools associated with schoolwide student performance in ELA and math in concurrent years, after controlling for other factors?
  • Are the CLASS observation scores of the low-performing schools associated with student growth in achievement in ELA and math, after controlling for other factors?
  • Are changes in low-performing schools' CLASS observation scores associated with changes in growth in student achievement in ELA and/or mathematics, after controlling for other factors?

Study Design: REL-NEI researchers will first conduct a series of descriptive analyses of the low-performing schools' average CLASS domain and dimension scores. Next, researchers will use regression models to explore how CLASS domain and dimension scores are related to overall school-level achievement and school-level student growth in achievement in ELA and math during the concurrent school year, while controlling for school characteristics (e.g., percentage of English learner students, percentage of students eligible for free or reduced-price lunch, percentage of students with disabilities, and students' race/ethnicity). Finally, REL Northeast & Islands will examine how changes in average schoolwide CLASS scores from one year to the next are associated with changes in student growth.

Projected Release Date: Fall 2019

Partnership or Research Alliance: Massachusetts School Turnaround Research Partnership

Related Products: What's Happening report, Stated Briefly report

Principal Investigators & Affiliation:
Susan Bowles Therriault, E.D., American Institutes for Research
Jill Walston, Ph.D., American Institutes for Research