Co-Principal Investigators: Hank Fien and Deanne Crone (University of Oregon)

Purpose: This project will assess the implementation and impact of a targeted middle grades intervention to improve reading and school engagement among students with reading difficulties taking place within a universal intervention to improve reading and engagement among all students.

Project Activities: The project will determine the impact of the intervention on students who receive the intervention in 6th grade, due to their 5th grade reading scores, and students who receive the intervention in 7th and 8th grades, due to their 6th grade reading scores. One cohort of students will be followed from 6th through the middle of 10th grade and the academic and school engagement outcomes of the treatment students will be compared to the control students. Using a regression discontinuity design, the comparison will primarily involve students who score just below the cutoff point on the reading performance measures with students who scored just above the cut score. To help interpret the findings, the project will also determine the fidelity of implementation of the intervention.

Products: This work will produce published reports regarding the impact of the intervention on a range of academic and behavior outcomes. In addition, participating districts will receive information on the implementation and impacts of the intervention through within-district meetings held regularly throughout each academic year and three across-district meetings, held in Years 2-4. The latter will also include professional development for district personnel on analyzing and interpreting the implementation and student performance data.

Setting: Five public school districts in western Oregon.

Population: One cohort of students starting 6th grade in the fall of 2010 in 55 schools (37 elementary and 18 middle schools) in 5 districts in western Oregon.

Intervention: All schools are implementing a universal approach to improve student academic achievement and engagement, and both treatment and control students will take part in the universal support systems offered at their school. A second targeted intervention will be provided to students identified as having reading difficulties and it will be composed of a reading intervention and an intervention to improve their engagement with schools. The reading intervention provides a minimum average of 20-30 minutes of explicit reading instruction per day, using a program that is distinct and different from programs or instruction received by students in the control group. The reading instruction is to be individualized through the use of screening and diagnostic assessments. The engagement intervention links each student to an adult advocate who provides daily, weekly, or monthly feedback to the student. This feedback is to be individualized through the use of indicators of psychological and behavioral school engagement. In addition, the interventions will be monitored and adjusted through data-based decision-making. School-based data teams will meet formally at least once per academic term (minimum of three times per year) to review reading achievement and engagement data and identify students who require more intensive support as well as to identify modifications in the overall reading and engagement interventions.

Research Design and Methods: The project will use a regression discontinuity design to evaluate the impact of the intervention on student engagement and achievement. Students will be assigned to the treatment group in the 6th grade if they score below a cut point on their combined performance of 5th grade Oregon Assessment of Knowledge and Skills (OAKS) reading assessment and a measure of passage reading fluency. Students will be assigned to the treatment group for the 7th and 8th grades if they score below the cut point on their combined performance on 6th grade OAKS reading assessment and the Spring 6th grade passage reading fluency assessment. Student academic and engagement outcomes will be compared in each year of middle school. In addition, students will be followed to the middle of 10th grade and high school outcomes will be compared.

Control Condition: Control students receive the universal academic and engagement approach, as do the treatment students, but not the targeted intervention. Students scoring above the cut point on the combined 5th grade OAKS reading assessment and passage reading fluency assessment will not receive the treatment in 6th grade. Students scoring above the cut point on the combined 6th grade OAKS reading assessment and passage reading fluency assessment will not receive the treatment in the 7th and 8th grades.

Key Measures: The project will use several sets of measures for student achievement and engagement outcomes, and implementation fidelity. Student achievement will be measured using the OAKS assessments in reading, math, and science; GPA in academic courses, the number of academic courses taken and the number failed, and an oral reading fluency assessment. Student engagement will be measured using attendance, tardiness, discipline referrals, dropping out, and the Student Engagement Instrument will be used to survey students twice a year in middle school and once a year in high school.

Fidelity of the reading intervention will be measured in two ways. The Explicit Learning Observation Tool will be used three times a year in grades 6 to 8 in all classrooms to determine what and how reading is being taught in the treatment and control classrooms. In addition, teacher logs documenting lesson plans, materials, and instructional strategies will be collected from the schools three times per year. Fidelity of the engagement intervention will be measured using the formal documentation required including the Individual Student Engagement Plans and, when available (based on the type of engagement intervention employed), the daily report forms or the weekly progress reports for each treatment student. Data-based decision-making will also be measured through observation of data teams three times each year, and by formal documents including meeting frequency, topics, and adjustments in the overall intervention and adjustments for specific students.

Data Analytic Strategy: The impact evaluation will use a multi-level model of students nested within schools with random effects allowing for variation by school. For the high school outcomes, the model will be extended to allow random variation in both middle school attended and high school attended, as middle schools are not strictly nested in high schools. Exploratory mediator analysis will be done to suggest mechanisms through which impacts occur using structural equation modeling.

Project Website: http://ctl.uoregon.edu/content/msip-home

Products and Publications

Journal article, monograph, or newsletter

Crone, D. A., Carlson, S. E., Haack, M., Kennedy, P. C., Baker, S. K., & Fien, H. (2016). Data–Based Decision–Making Teams in Middle School: Observations and Implications from the Middle School Intervention Project. Assessment for Effective Intervention, 41(2): 79–93.

Fien, H., Anderson, D., Nelson, N.J., Kennedy, P., Baker, S.K., and Stoolmiller, M. (2018). Examining the Impact and School-Level Predictors of Impact Variability of an 8th Grade Reading Intervention on At-Risk Students' Reading Achievement. Learning Disabilities Research & Practice, 33(1), 37–50.

Nelson, N.J., Smith, J.L.M., Fien, H., Crone, D.A., Baker, S.K., and Kame'enui, E.J. (2016). Increasing School Completion by Providing Comprehensive Support in the Transition Years: Lessons Learned From the First Year of the Middle School Intervention Project. Journal of Special Education Leadership, 29(1), 1–13.