Co-Principal Investigator: Rebecca Silverman and Jeffrey Harring (University of Maryland, College Park)

Purpose: In a prior NCER funded grant (R305A090152, Investigating Vocabulary Breadth and Depth and Comprehension in English Monolingual and Spanish-English Bilingual Elementary School Students), researchers explored the relationships between linguistic and component reading skills and reading comprehension in English monolingual students and Spanish-speaking English Language Learners (ELLs). Findings from this study suggest that skills in linguistic awareness are related to reading skills. Linguistic awareness includes knowledge of components of language, including semantics, morphology, and syntax. ELLs typically lag behind their native English-speaking peers in linguistic awareness skills. In this project, the team will develop and evaluate the feasibility and potential efficacy of CLAVES (Comprehension, Linguistic Awareness, and Vocabulary in English and Spanish), a supplemental intervention for fourth grade Spanish-speaking ELLs who are acquiring language and literacy in English. In addition to explicit, text-based instruction in semantics, syntax, and morphology, CLAVES will include collaborative discussions with ELL peers and ongoing English learner supports. CLAVES will be designed for use by any teachers with specialized training in teaching ELLs.

Project Activities: In the first two years of the project, the team will complete four cycles of drafting, testing, and revising CLAVES. Working groups of teachers and students as well as expert advisors will provide feedback and suggestions for improvement on CLAVES. In Year 3, the researchers will carry out a pilot study to assess the promise of CLAVES to improve ELLs’ linguistic awareness and reading comprehension skills.

Products: At the conclusion of the project, the team will have produced a fully developed version of CLAVES for fourth grade Spanish-speaking ELLs, including a curriculum guide and a teacher manual. The project team will disseminate findings through peer-reviewed publications, publications for practitioners, and presentations at conferences for researchers and practitioners.

Structured Abstract

Setting: The research will take place in 6 urban and semi-urban schools in Massachusetts and in Maryland. Participating schools serve from 37 – 60% Latino students, and 17% to 34% ELLs. Most ELLs are Spanish-speaking in the schools.

Sample: In Years 1 and 2, a total of 48 Spanish-speaking students in fourth grade identified as ELLs by the school will participate, along with 18 teachers. These participants will engage with Boston College and University of Maryland researchers in design studies. In Year 3, a small randomized controlled pilot study will be carried out with 96 students.

Intervention: The CLAVES intervention will be aligned both with the Common Core State Standards and the World Class Instructional Design and Assessment (WIDA) English Language Proficiency Standards. CLAVES will include explicit instruction on linguistic awareness skills (i.e., semantics, morphology, and syntax), collaborative discussion sessions, and ELL supports. Collaborative discussions provide an opportunity for ELLs to engage in authentic, text-based conversations with their ELL peers about big ideas that arise within readings. Students are asked to take a position on a controversial question posed by the text and then discuss their position with their peers. Supports for ELLs will include pictures, graphic organizers, and multimedia elements to provide further access to text that ELLs may find challenging. CLAVES will include four topical units of four weeks each with four daily 30-minute lessons per week. Students will be divided into high and low groups based on English language proficiency (ELP), and two sets of texts (one suitable for those with lower ELP and one for those with higher ELP) will be selected for each unit.

Research Design and Methods: The development and field-testing stage in Years 1 and 2 will include formative experiments with two cycles each year. Each cycle will include eight stages of development (drafting, advisory board feedback, revising, teacher feedback, revision, field-testing, teacher, student and research assistant feedback, and revisions). In the first two years, CLAVES will be administered by research assistants to maximize fidelity. Teacher Working Groups will meet monthly to provide feedback on development activities, and teachers will also be individually interviewed. Student Working Groups will also provide input for revision of CLAVES. Qualitative data gathered from meeting notes, field notes, and expert input will be coded for recurrent themes. Case studies will be constructed within schools to position findings within the specific school context. Fidelity of implementation will be assessed through observations and a checklist. In Year 1, researchers will develop and administer a new assessment called the Linguistic Awareness Skills Test (LAST) that is closely aligned to the intervention and will measure semantic, morphological, and syntactical awareness.

In Year 3, the team will carry out a small-scale pilot study in which ELL students will be randomly assigned within classrooms to participate in the treatment or control group. CLAVES will be administered by ELL teachers. Standardized assessments of linguistic awareness and reading comprehension in English will be administered at the beginning and end of the study. A limited set of Spanish language and literacy assessments will also be administered.

Control Condition: Students in the control condition in the pilot study will receive typical instruction.

Key Measures: The Assessing Comprehension and Communication in English State-to-State for English Language Learners Assessment (ACCESS) for ELLs will be used to place students in lower or higher reading level groups. The pilot study will include the following standardized measures of word level and reading comprehension skills: Woodcock-Munoz Language Survey-Revised, Clinical Evaluation of Language Fundamentals (Fourth Edition), Extract the Base assessments (ETB), Gates-MacGinitie Reading Test (Fourth Edition), and the Test of Sentence Reading Efficiency and Comprehension (TOSREC).

Data Analytic Strategy: Qualitative data will be analyzed two ways: (1) coding and triangulation of data from interviews, observations, and field notes; and (2) development of case studies to identify strengths and opportunities for improvement in the intervention. Researchers will answer questions of feasibility and fidelity by identifying themes (at the word, phrase, or larger chunk of text level) in qualitative data sources and coding them for content. Cross-case analyses will identify salient case features and how these features compare across cases. Quantitative data in the pilot study will be analyzed using multi-level structural equation modeling to determine intervention effects on linguistic awareness and reading comprehension. Discourse analysis from the discussion sessions will be used to evaluate changes in treatment students’ language use during the intervention period.

Project website: https://clavescurriculum.org/

Products and Publications

Journal article, monograph, or newsletter

Harring, J.R., Weiss, B.A., and Li, M. (2015). Assessing Spurious Interaction Effects in Structural Equation Modeling. Educational and Psychological Measurement, 75(5): 721–738.