Evaluation of the Quality Teaching for English Learners Program

Regional need and study purpose. While all areas of the country have experienced rapid growth in the number of English language learner students, the West Region leads the country, with 57 percent of all English language learner students enrolled in its elementary and secondary schools. Subject-area teachers report feeling overwhelmed and unable to teach students who are years behind native English speakers in reading and writing. This study, which uses an intent-to-treat model, evaluates the effects of offering middle schools teachers in eight Southern California school districts the opportunity to participate in Quality Teaching for English Learners (QTEL), a professional development model for secondary teachers aimed at improving the quality of instruction for English language learner students. The main purpose of the study is to evaluate the effects on teacher knowledge and attitudes, teacher practices, and students' English reading and writing skills using an experimental design. The study uses an intent-to-treat model, which means it studied the effects of offering teachers the professional development, not the effects of the program itself.ssss

Intervention description. The QTEL program offered in this study focuses on helping teachers develop the expertise to support adolescent students as they develop their ability to read, write, and discuss academic texts in English in rigorous academic courses. The QTEL professional development model consists of three components: a seven-day summer professional development program, an intensive coaching program during the school year, and collaborative lesson planning meetings. To maximize the potential impact on a single cohort of students progressing through middle school, the primary group of teachers targeted by the intervention consists of grade 6 teachers in 2007/08, grade 7 teachers in 2008/09, and grade 8 teachers in 2009/10.

Study design and period. The study is an experimental trial. Eight Southern California districts agreed to participate and to identify qualifying schools (schools in which English language learner students made up at least 10 percent of the population). The final sample consisted of 52 schools. QTEL was offered to English language arts and English language development teachers in 26 middle schools, randomly selected from the study sample. English language arts and English language development teachers in the other 26 schools constitute the control group. The study will measure impacts on teacher knowledge and attitudes, classroom practices, and student achievement over the three-year period 2007/08–2009/10.

Key outcomes and measures. The study will assess three outcome domains: teacher pedagogical knowledge and attitudes, classroom practices, and student achievement. Teachers' pedagogical content knowledge associated with the key delivery components of the QTEL model and teachers' attitudes toward the model will be measured through surveys. The quality of classroom instruction experienced by English language learner students will be assessed through classroom observation using the Sheltered Instruction Observation Protocol (SIOP) and the Program Aligned Classroom Observation (PACO), an instrument designed to capture specific constructs included in the QTEL professional development. The student achievement outcome will be assessed using scores on the California Standard Test in English language arts for all students in all participating schools and scores on the California English Language Development Test for all English language learner students who take that test.

Data collection approach. Data will be collected from all teachers and students in the appropriate grades in participating schools. Implementation data (attendance records capturing teacher participation in professional development activities) are being collected in 2007/08, 2008/09, and 2009/10. Teacher/classroom outcome data (capturing pedagogical content knowledge, attitudes, and classroom practices) and student achievement data (English language arts and English language proficiency test scores) are being collected in 2006/07, 2007/08, 2008/09, and 2009/10. This means that baseline data (from 2006/07) are available for teacher/classrooms and student achievement outcomes.

Analysis plan. . The impact analyses will examine the effects of offering QTEL on teacher knowledge and attitudes, classroom practices, and student achievement outcomes using hierarchical linear regression models to account for the effect of data clustering by school and classroom. With 52 schools and a minimum of 400 students and 100 English language learner students per grade, the projected sample size is sufficient to detect effect sizes of 0.22 standard deviation units on test scores for all students and 0.23 for English language learner students. An effect size of this magnitude is equivalent to about one year of growth on norm-referenced standardized tests in reading (Hill et al. 2008). The estimated minimum detectable effect size for teacher and classroom outcomes is 0.40 standard deviation units on teacher knowledge assessments and SIOP/PACO classroom observations.

Principal investigators. Johannes M. Bos, Ph.D., Berkeley Policy Associates, and Raquel C. Sanchez, Ph.D., Berkeley Policy Associates.

Additional Information. Region, contact information, and references.

Click here for full printable version