Project Activities
During the first phase of the project, the researchers will conduct three studies focused on iteratively developing and revising the intervention, attending to feasibility and usability for teachers and students. This work includes collaboration with regional teacher collaborators (RTCs) and an advisory board that will provide input on the fully drafted intervention and professional development (PD) workshops. The research team will also develop student learning measures using existing assessments. In the second phase, the researchers will develop three PD workshops with the RTCs and will develop and test a fidelity of implementation measure. Then, they will recruit new RTCs to field test the fully drafted intervention and PD workshops. In the final phase of the project, the team will conduct a randomized controlled trial of the fully developed intervention to estimate preliminary evidence of effects of Words as Toolson student learning.
Structured Abstract
Setting
The project will take place in middle school ESL classes serving EB students at intermediate and advanced levels of English proficiency in the Pittsburgh public schools in Pennsylvania and Washoe County school district in Nevada.
Sample
Three iterative development cycles will involve 5 teachers and 75 students in Pennsylvania and 5 teachers and 65 students in Nevada. The pilot study will take place in 18 middle schools (n = 36 teachers, 468 students) in Nevada. The intended end-users of this intervention are middle school ESL teachers.
Words as Tools will be organized into three units informed by the NGSS Disciplinary Core Ideas: life science, earth and space systems, and physical science. Each unit will comprise a series of mini investigations. Within each unit, students will engage in lessons to develop metalinguistic awareness for academic vocabulary learning in science. Students will have opportunities to (1) analyze high-frequency Greek and Latin roots, (2) construct related words using derivational suffixes, and (3) resolve polysemy of word meanings. Building knowledge of these language components supports flexible, generative word knowledge because each of these applies to multiple words in science and other domains. The researchers will select the targeted morphemes and polysemous words from essential academic words in science, which include both discipline specific science words and general academic words. Lessons across all 3 units will engage students in robust vocabulary instruction of a total of 48 target high-frequency words in science, 48 target Greek/Latin high-frequency science morphemes, 18 target high-frequency derivational suffixes in science, and 36 target polysemous words. The researchers will design the lessons to help students build knowledge of essential science words as well as how words work in science. Furthermore, because EBs should have opportunities to draw on their multilingual resources, all intervention components will be designed with a lens of culturally sustaining pedagogy.
The Words as Tools intervention builds on a previous IES development project, English Learners' Robust Academic Vocabulary Encounters (EL RAVE). EL RAVE demonstrated that explicit instruction about morphological analysis equipped EBs with metalinguistic skill to learn new words, but EL RAVE was limited to Latin roots. Words as Tools amplifies this focus by teaching multiple aspects of metalinguistic awareness at once. Moreover, Words as Tools situates metalinguistic instruction in science content related to the Next Generation Science Standards (NGSS), thus addressing urgent needs for development of both language and science concepts.
Research design and methods
During the first phase of the project, the researchers will conduct three iterative design studies focused on development and revision of the intervention and will thematically analyze qualitative data sources to address questions of feasibility and usability for teachers and students. They will engage in collaborative development and field testing of Words as Tools units with regional teacher collaborators(RTCs) in each district. The six RTCs will advise integration of professional development (PD) supports into intervention materials. The research team will also develop student learning measures using existing assessments as blueprints. These will be field tested to estimate reliability and validity and will be revised as needed. In the second phase of the project, the team will develop and pilot test a fidelity of implementation measure. In addition, the research team will develop three PD workshops with feedback from the RTCs. The researchers will recruit four new RTCs who have no familiarity with the intervention to field test the fully drafted intervention and PD workshops. A national teacher advisory panel will also advise on development of the intervention materials and PD supports and the relevance of these materials for a diversity of regional contexts. In the final phase of the project, the team will conduct a randomized controlled trial (RCT) of the fully developed intervention to estimate preliminary evidence of effects of Words as Toolson student learning. They will use a randomized block design, assigning half the classes in the pilot study to the intervention condition.
Control condition
The intervention will be compared to a closely monitored business as usual comparison condition of ESL classes. The researchers will document curricular topics and instructional practices in BAU classes through survey responses, direct observations of ESL lessons, and on-line instructional log entries provided during the intervention period (60 days).
Key measures
The researchers will use interviews and surveys with teachers and students, as well as observations and analysis of lessons, to inform the iterative design of the intervention and to assess feasibility and usability. They will adapt existing measures for new science content to assess student learning and will also administer a continuum of measures to capture proximal, distal, and far-transfer effects. The team will develop a fidelity checklist for use during the pilot study for implementation fidelity.
Data analytic strategy
The research team will use qualitative analyses in for intervention design and revision, and correlational techniques. For the RCT, they will use multilevel modeling employing a 3-level random-intercept model (to account for the nested data structure) to estimate treatment effects for proximal, distal, and far transfer measures.
Cost analysis strategy
The research team will document and calculate all resources used in intervention and control conditions to estimate intervention cost per student and classroom.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Researchers will produce a fully developed Words as Tools intervention for EB adolescents, a cost analysis of the intervention, and a project webpage for teachers, administrators, and other school-based professional with sample intervention materials. The team will also engage in dissemination activities throughout the project that includes dissemination to the research community including through peer-reviewed publications for both scientific and practitioner audience, to the broader practitioner community, and to partner school districts.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
