Project CRISS Reading Program and Grade 9 Reading Achievement in Rural High Schools
Key outcomes and measures
The sole outcome measure is student reading comprehension. The comprehension subtest of the Stanford Diagnostic Reading Test (SDRT), 4th edition, has been selected as the standardized measure across states to assess student outcomes. The SDRT-4 is a group-administered reading assessment designed to pinpoint student strengths and weaknesses in reading. The comprehension subtest will be used with items that measure the understanding of three kinds of text: recreational reading, informational reading, and reading applied text from everyday situations.
Descriptive data on how faithfully Project CRISS is implemented by schools and teachers will be collected in the treatment schools, with some parallel measures in the control schools. Two short online questionnaires will be administered to grade 9 teachers and principals in both treatment and control schools. Questions on teacher and principal background information, such as years of experience and highest degree held, will be included to describe general staff characteristics at the schools and determine whether there are differences between treatment and control schools. The questionnaires also ask about schoolwide professional development programs related to reading comprehension other than Project CRISS. Four questions ask the extent to which teachers and principals participate in Project CRISS-recommended activities in the treatment school (conducted by a Project CRISS local facilitator) or parallel activities in the control schools that might be organized by an in-school literacy coach. These measures will help compare Project CRISS schools with the business-as-usual control schools.
A main source of implementation data in the treatment schools will be a monthly electronic log of specific activities. The Project CRISS local facilitator will be asked to report on Project CRISS activities, such as meetings, coaching, and other activities supposed to occur in addition to day-long workshops conducted by the Project CRISS national trainer. The monthly data will be examined across two school years of Project CRISS professional development to construct an index of how well the program was implemented. Attendance data at training events and other documented information from the Project CRISS national trainer will also be gathered to assess whether implementation was full or partial.
Classroom observations will be used to compare teacher behaviors related to the learning principles and strategies that are the basis of Project CRISS in subsample of treatment and control schools. Because the cost of sampling an adequate number of teachers to estimate accurate teacher impacts is prohibitive, teacher behavior is treated as an implementation variable rather than an impact measure. The observational results will describe the extent to which research-based learning principles espoused and taught through Project CRISS are visible in schools after two years of Project CRISS training and technical assistance. Again, the Project CRISS schools will be compared with the business-as-usual control schools.
A version of the Vermont Classroom Observation Tool (VCOT) is being used because it measures teacher behaviors that are very similar to those expected from Project CRISS. This instrument was developed for and used in other IES research, though some minor adaptations were made for this study with the consultation of the developer. All observers will receive intensive training and certification by the VCOT developer.