|Title:||English Learner Vocabulary Acquisition (ELVA): Promoting the Vocabulary and Language Proficiency of Spanish Speaking English Learners in Second Grade|
|Principal Investigator:||Luft Baker, Doris||Awardee:||Southern Methodist University|
|Program:||Policies, Practices, and Programs to Support English Learners [Program Details]|
|Award Period:||3 years (8/1/2014 – 7/31/2017)||Award Amount:||$1,499,586|
|Type:||Development and Innovation||Award Number:||R305A140471|
Co-Principal Investigator(s): Stephanie Al Otaiba; Ron Cole and Wayne Ward (Boulder Language Technologies)
Purpose: The goal of this project was to create and test an intelligent tutoring system called English Learner Vocabulary Acquisition (ELVA) that increases the vocabulary knowledge, text comprehension and English language proficiency of Spanish-speaking English Learners (ELs) in second grade. The project focused on academic words that are necessary to describe, explain and discuss science phenomena and social studies issues relevant to second grade content that is included in the Common Core State Standards, the Next Generation Science Standards, the Social Studies standards, and the Texas Essential Knowledge and Skills Standards.
Project Activities: Researchers iteratively developed, refined, and tested the vocabulary activities included in the system. In the first two years of the project, researchers developed the content for six ELVA modules (3 in science and 3 in social studies). While the content team developed the units and activities, the technology team developed the speech recognition system and programmed the virtual tutor to react to student potential responses to different activities. To build the system, technology researchers used excerpts from students engaging in a "Questioning the Author" exercise with a human tutor. In each iteration, researchers also elicited feedback from teachers and students on the passages and functionality of ELVA. In Year 2, researchers investigated the feasibility of incorporating ELVA into classroom instruction. In Year 3, researchers conducted a pilot study to determine the promise of the intervention to develop student vocabulary knowledge and understanding of science and social studies topics.
Products: The products of this project include a fully developed intelligent-tutoring system, ELVA, for second-grade Spanish-speaking ELs for seven units, and teacher training for using ELVA in the classroom. The project provided evidence of the usability, feasibility, and fidelity of the ELVA implementation. Researchers also produced peer-reviewed publications, created a video of the project, and presented results in multiple conferences for researchers and practitioners.
Setting: This project took place in public and independent schools in Texas with a large percentage of Latino students (i.e., above 77%). All the schools were located in urban settings, and they had varying percentages of students who were on free and reduced lunch.
Sample: All studies included second-grade ELs. Studies in the development phase included 30 students and 6 teachers. The feasibility study included 60 ELs in approximately 12 classrooms. The pilot study included 217 ELs in 26 classrooms. Students had different levels of English language abilities.
Intervention: The ELVA app was developed for PC computers. The content of the intervention included approximately 18 passages of 250ndash;300 words related to themes in social studies (e.g., Heroes, Government, Geography) and science (e.g., Earth Science, Organisms and the Environment). Researchers selected 5ndash;6 target words per passage for in-depth teaching (e.g., admire, polite, survive, habitat) and 1 or 2 topics to build student linguistic and syntactic knowledge for each theme (e.g., recognizing morphemes such as prefixes and parts of speech). Target word selection was based on Beck & McKeown's definition of Tier 2 and Tier 3 words, and feedback from teachers and expert vocabulary and EL researchers. Comprehension activities used a Questioning the Author approach. The passages and activities were included in an interactive unit composed of five sessions each. In the first session the virtual tutor, Eva, read the passage aloud containing the target words. During the read aloud, Eva engaged students in a dialogue around the content of the passage that included the use of the target words. ELs then worked through a series of activities designed to facilitate deep learning of the target words, their associated concepts, and linguistic realizations with Eva's guidance. At the end of the week, Eva read the passage again and engaged students in a deeper conversation that elicited explanations in which Eva facilitated the use of the words and the linguistic features students had learned during the week and in previous units. Teacher professional development focused mainly on helping teachers troubleshoot problems students might have had with the app.
Research Design and Methods: Researchers used an iterative design-test-refine process to produce intuitive and engaging activities for the ELVA app. After each passage was written, graphic information was added. Passages and activities were then tested with either a human tutor reading the passages or an electronic voice. The ELVA system recorded student responses and a speech recognizer analyzed these responses in comparison to expected answers. Based on student responses, the passage and activities were revised and retested with students until ELVA functioned as expected. Information for improvement of ELVA was obtained from student interviews, student language samples, student logs in the computer system, and observations of students using ELVA.
To test the usability and feasibility of ELVA, researchers conducted observations, analyzed information from teacher and student surveys and interviews, and downloaded data from the app on student language productions when using the system. This information was then summarized descriptively.
To determine the promise of ELVA to change academic outcomes, researchers used a clustered randomized design. Classrooms (N = 26) were first matched within schools based on the number of eligible students per classroom, and program of instruction (i.e., whether instruction was provided in a bilingual setting or in an English only setting). Classrooms were then randomly assigned to the treatment or control condition. There were 112 students in the treatment condition and 105 students in the control condition. Both groups of students had similar demographic characteristics.
Control Condition: In the control condition in the pilot study, 105 second grade ELs in 13 classrooms received standard classroom science and social studies instruction. All teachers in the control condition also received a list of the words taught in the treatment group.
Key Measures: To determine the usability and feasibility of the program, researchers used surveys, observations, and interviews with teachers and students. To iteratively develop the content and technology of the program, researchers used the same measures as above and also the data collected through the system.
In the pilot study, researchers used the Depth of Vocabulary Knowledge (DOK) in Spanish and in English. This measure was researcher-developed and adapted from other vocabulary projects. Students were allowed to respond to questions in Spanish, in English, or in both languages. To determine the quality of student responses, selected language samples were coded qualitatively. In addition, researchers used the Test of Oral Language Development-Primary, Third Edition (TOLD-P3); DIBELS Oral Reading Fluency (DORF); the Gates-MacGinitie Test of Reading Comprehension, and the Receptive One Word Picture Vocabulary Test (ROWPVT), and the Expressive One Word Picture Vocabulary Test (EOWPVT) bilingual edition.
Data Analytic Strategy: Descriptive data from surveys, classroom observations, and student performance were used to iteratively revise ELVA and determine feasibility. For the pilot study, researchers used a multilevel model with two levels: students and classrooms. A school level was not included because the between school variance was very small (ICC = .001%).
Related IES Projects: Measuring the English Language and Vocabulary Acquisition of Latinx Students (Project MELVA-S) (R305A200521); Project Early Vocabulary Intervention (R324A110135)
Publications and Products
Peer Reviewed Articles
Baker, D.L., Ma, H., Polanco, P., Conry, J., Kamata, A., Al Otaiba, S., Ward, W., and Cole, R. (2020a). Development and Promise of a Vocabulary Intelligent Tutoring System for Second-Grade Latinx English Learners. Journal of Research on Technology in Education. DOI: 10.1080/15391523.2020.1762519
Sano, M., Baker, D.L., Collazo, M., Le, N., and Kamata, A. (2020). Measuring the Expressive Language and Vocabulary of Latino English Learners Using Hand Transcribed Speech Data and Automated Scoring. International Journal of Intelligent Technologies and Applied Statistics, (13)3, pp. 229ndash;255, https://doi:10.6148/IJITAS.202009_13(3).0003.
Additional online resources and information:
Baker, D. L. (May 2018). Three-minute video on the English Language Vocabulary Acquisition Project http://stemforall2018.videohall.com/presentations/1202.