Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Improving the Quality of English Language Arts Teaching through the Use of an Observation Protocol
Center: NCER Year: 2011
Principal Investigator: Grossman, Pamela Awardee: Stanford University
Program: Teaching, Teachers, and the Education Workforce      [Program Details]
Award Period: 7/2011–6/2014 Award Amount: $1,454,478
Type: Development and Innovation Award Number: R305A110864

Co-Principal Investigators: Susan O'Hara and Susanna Loeb

Purpose: In ongoing efforts to improve student learning and the quality of instruction, practitioners and education stakeholders rely on measures of classroom practice. Researchers have been developing classroom observation tools to evaluate teacher performance, such as the Protocol for Language Arts Teaching Observation (PLATO), which highlights 12 elements of high quality teaching in English language arts (ELA). Although the initial studies demonstrated the power of PLATO as a research tool for observing teaching, this proposal seeks to develop and test the use of PLATO as a tool for professional development to improve the classroom practices of teachers of ELA.

Project Activities: Using an iterative design, the researchers will design a professional development program for middle school ELA teachers. During the first year, researchers will develop this new model of professional development, based on targeted instructional elements of PLATO. After initial development is complete, the researchers will implement the model with a small group of teachers. In Year 2, researchers plan to run a pilot study with a larger group of teachers in order to test the promise of this model to improve classroom practice and student achievement in reading and writing. Teachers participating in the PLATO professional development will be compared to a group of teachers and their students not participating in the professional development. In Year 3, the research team will conduct follow-up observations to trace on-going effects of the professional development (if any), in addition to completing analyses and writing up of project data.

Products: Products from this project will include a fully developed and validated professional development framework that builds off of PLATO. Peer-reviewed publications will also be produced.

Structured Abstract

Setting: The research will take place in middle schools in the San Francisco, California with high proportions of students from non-dominant cultures and linguistic communities (90 percent are of color; 30.5 percent are English language learners and 55.5 percent qualify for free or reduced lunch).

Population: Middle school English language arts teachers will participate.

Intervention: The professional development will involve successive cycles of observation, goal-setting, learning about PLATO elements and how to enact them successfully, deliberate practice of classroom moves associated with these elements, and targeted feedback on observed practice. Researchers will provide teachers with modified PLATO training and significant experience scoring videos of classroom practices and giving feedback based on the elements. Following this training, teachers will be provided with the baseline data on the classroom practices at their school and across the schools in the study. Each school team will identify two to three elements to work on during the academic year, based both on their school data and on the relationship of elements to student outcomes. Over the academic year, teachers will receive bi-monthly professional development around the specific elements they have chosen to work on. The professional development will include iterative cycles of teachers trying out new classroom moves and then trying out these approaches in their classrooms. Teachers will videotape their efforts and share these videos with team members and receive feedback on their efforts.

Research Design and Methods: In the initial year of the project, the team will develop materials to support the use of PLATO as a professional development tool. Once the initial development process is complete, researchers will implement the professional development model with 16 English teachers from 4 middle schools. The researchers will document the nature of teachers' professional development experiences through field notes, audio-recorded observations, interviews with teachers and administrators, and focus group sessions. Documentation data will be analyzed to determine the level of support needed by teachers to understand PLATO elements and to begin to incorporate them into their classroom practice. This data will also be analyzed to study the challenges teachers encountered in understanding and using PLATO. This data will be used to test the intervention and provide feedback to support revision of the model.

In Year 2, the researchers will run a pilot study with an expanded number of about 32 teachers from the middle schools participating in the first year of the project. They will attend a summer institute and revise activities for the academic year, based on experiences in Year 1. Teachers at each school will again identify specific elements of PLATO to work on, working on the same elements or different elements as the year before. The results of the PLATO training with the teachers and their students will be compared to a set of teachers in non-participating schools. In the final year of the project, the research team will complete follow-up observations to trace any ongoing effects of participating in the PLATO professional development activities on teacher practice and student outcomes.

Control Condition: For the pilot study in the second year, approximately 8 teachers at each of 4 new middle schools (about 32 in all) will serve as a comparison group. These teachers will receive only the professional development courses they normally would, and none will receive training on or with PLATO, but they will be observed three different times across the school year, and again in Year 3.

Key Measures: The primary instruments for measuring classroom practice will be the PLATO tool and another more general teacher practice instrument. These measurements will be collected through observations of the teachers at regular intervals across the 3 years of the grant. They will also collect student-level data, including standardized test scores and writing performance assessments. In particular, they will collect scores on the California Standards Tests' English Language Arts subset and will use writing assessments such as ERB's Writing Assessment Program (WrAP). The writing samples will be collected at the fall and spring from both participating and comparison classrooms.

Data Analytic Strategy: Researchers will conduct a dependability study to model the reliability of the PLATO rubric. They will use regression and hierarchical modeling techniques to analyze both the data on teacher performance and the data on student measures. The regressions will help control for initial scores of the teachers and initial characteristics of student and the school context.

Products and Publications

Journal article, monograph, or newsletter

Hill, H.C., and Grossman, P. (2013). Learning From Teacher Observations: Challenges and Opportunities Posed by new Teacher Evaluation Systems. Harvard Educational Review, 83(2): 371–384.

** This project was submitted to and funded under Teacher Quality: Reading and Writing in FY 2011.