|Title:||Making Individualized Literacy Instruction Available to All Teachers: Adapting the Assessment to Instruction (A2i) Software for Multiple Real-World Contexts|
|Principal Investigator:||Connor, Carol M.||Awardee:||University of California, Irvine|
|Program:||Education Technology [Program Details]|
|Award Period:||4 years (7/1/13-6/30/17)||Award Amount:||$1,499,994|
|Goal:||Development and Innovation||Award Number:||R305A160404|
Previous award number: R305A130517
Previous University Affiliation: Arizona State University
Purpose: The purpose of this project is to complete development of Assessment to Instruction (A2i) software, which is an integral part of the Individualizing Student Instruction (ISI) intervention. A2i was originally developed and funded under an earlier IES grant (Child Instruction Interactions in Early Reading: Examining Causal Effects of Individualized Instruction) as a research tool to compute recommended amounts and types of instruction taking into account that the effect of specific types of reading instruction appears to depend on students' language and reading skills. Although, under an earlier IES funded efficacy and replication grant (Child-Instruction Interactions in Reading: Examining Causal Effects of Individualized Instruction in Second and Third Grade), it shows potential as an effective way to support teachers' implementation of effective differentiated reading instruction. In its current form it is not feasible for educational leaders (e.g., literacy coaches, principals, district administrators) and teachers to use A2i without substantial amounts of researcher support. Therefore, this study aims to develop a usable and scalable version of A2i.
Project Activities: Following a design-based implementation research approach, the team will carry out an iterative design, development, and testing process in two phases. In Phase one, the team will re-design and improve the A2i software with input from teachers and educational leaders, as well as the developers of A2i and ISI. In Phase two, the team will complete a quasi-experimental test of the improved A2i software with teachers and schools who have not previously used the A2i software.
Products: Products from this project will include a packaged and distributable version of A2i that can be easily installed and maintained by districts who wish to use A2i/ISI in the future. Peer-reviewed publications will also be produced.
Setting: Research will be carried out in three settings. ASU Preparatory Academy is a charter school located in downtown Phoenix with the goal of preparing all students for college and careers. University Academy in Panama City, FL, is also a charter school with similar aims. Serving economically and diverse student bodies, they will both be active collaborators in the design studies. The pilot study will be conducted in public schools in Girard, PA, which also serves a diverse student population and provides a more remote site, which offers the opportunity to better assess feasibility and promise.
Sample: The principal participants of the studies are the teachers, educational leaders, school/district administrators, and students at the three sites. The schools serve a highly diverse student body: across schools, approximately 40 percent of the students qualify for the Free and Reduced Lunch Program and about 79 percent belong to ethnic/racial minority groups. Approximately 12–16 teachers will participate in Year 1, an additional 16 teachers will participate in Year 2, and in Year 3, 36 teachers in 4 schools will participate in the pilot study.
Intervention: The A2i software is a web-based secure and password protected program designed to support educational leaders' and teachers' efforts to provide more effective and individualized literacy instruction from kindergarten through third grade. A2i explicitly links assessment results to recommendations for instructional strategies and activities, provides classroom organization and lesson planning tools, and online professional development, including video of expert teachers. Access and reports for administrators, literacy coaches, and other educational leaders will be developed as part of a package that schools will be able to implement without extensive and ongoing researcher support.
Research Design and Methods: The overall approach of this project is design-based implementation research. Work will be conducted in two phases that will allow us to test multiple iterations and improvements with teachers and educational leaders in close collaboration, as well as with teachers and educational leaders who are located farther away from the researchers, refining towards a "final" version of A2i that is usable by teachers and educational leaders. In the initial phase of research, the team will focus on the re-design and improvement of the A2i software. The team will test multiple versions of the A2i software in a rapid prototyping process. An additional goal of the redesign process is to examine how to best integrate progress monitoring assessment administration into A2i. In the second phase of the project, the team will carry out a quasi-experimental study with teachers and educational leaders who have not previously used the system. The pilot study will use a delayed treatment design with schools matched on school-wide percentage of students qualifying for the Free and Reduced Lunch Program, third grade performance on the state-mandated reading assessment, Title 1 status, and community (urban, rural, suburban).
Control Condition: The control schools will not receive training in or access to A2i and the ISI intervention until the final 3 months of the school year whereas the treatment group will begin training in the summer and will use A2i and ISI throughout the school year.
Key Measures: Teacher measures include Distance from Recommendation, which identifies how closely teachers' classroom practices match the ideal implementation of ISI; information about the amount of A2i use and the A2i resources used by individual teachers; and surveys and interviews. Student outcomes will be measured by researcher-developed measures (Word Match Game, Garden Path Maze), district specific assessment data, and classroom observations. During the redesign phase, the team will also rely on measures of the usability of A2i.
Data Analytic Strategy: Phase one of the study will rely on observational, interview, and log-file analytic techniques. To analyze outcomes from the quasi-experimental study carried out in Phase two, the team will use multi-level modeling with students nested in classrooms, nested in schools with the treatment condition entered at the classroom level with student Distance from Recommendation as the outcome. Because of limited power, they will examine effect sizes rather than relying solely on significant differences, and will examine how the effect sizes compare with effect sizes in their previous studies.
Related IES Projects: Child Instruction Interactions in Early Reading: Examining Causal Effects of Individualized Instruction (R305H040013) andChild-Instruction Interactions in Reading: Examining Causal Effects of Individualized Instruction in Second and Third Grade (R305B070074)