|Title:||Dynamic Support of Contextual Vocabulary Acquisition for Reading (DSCoVAR): An Intelligent Tutoring System|
|Principal Investigator:||Collins-Thompson, Kevyn||Awardee:||University of Michigan|
|Program:||Cognition and Student Learning [Program Details]|
|Award Period:||3 years (9/1/2014 - 8/31/2017)||Award Amount:||$1,500,000|
|Type:||Development and Innovation||Award Number:||R305A140647|
Co-Principal Investigator: Gwen Frishkoff (Georgia State University)
Purpose: Vocabulary knowledge is widely regarded as a precursor to skilled reading comprehension. Knowledge of words that are abstract or take on multiple meanings otherwise known as Tier 2 words, is often considered to be the key to higher-level comprehension. A complete grasp of the meaning of Tier 2 words may depend on reading the words multiple times across a variety of contexts. Tier 2 words take on greater importance in the intermediate grades (fourth through eighth grade) as students begin to read a wide range of texts, suggesting a need for specific interventions that target contextual word learning in those grades. The primary goal of this project is to develop and pilot test an intelligent tutoring system (ITS) designed to support the acquisition of Tier 2 words from written contexts.
Project Activities: The proposed research will develop and pilot-test Dynamic Support of Contextual Vocabulary Acquisition for Reading (DSCoVAR), a web-based intelligent tutoring system that will expose learners to new words in a variety of well-controlled contexts. DSCoVAR will use computational methods to estimate changes in partial word knowledge and will provide strategy training to promote effective use of context cues to meaning. The initial version of DSCoVAR will be developed in Year 1. System refinements and feasibility studies will occur in Year 2. Year 3 will be devoted to a pilot study evaluating the intervention using a randomized controlled trial design.
Products: The products of this project will be a fully developed ITS, DSCoVAR, developed and tested with sixth through eighth grade students. This tutoring system will address challenges in acquiring Tier 2 words from written contexts. Peer reviewed publications will also be produced.
Setting: Initial development and feasibility studies will take place in two middle schools (sixth through eighth grades), one located in an urban area in Georgia and the other in an urban area in Pennsylvania. The pilot study will take place during the after school programs of approximately five Title 1 middle schools located in urban areas in Georgia.
Sample: Initial development and feasibility studies will occur with approximately 250 participants from sixth through eighth grade. These students will be drawn from multiple schools in Georgia and Pennsylvania. The pilot study will include about 300 students from sixth through eighth grade, with equal numbers from each grade. Approximately 98% of students from schools that will participate in the pilot study are African American. The ratio of male to female students is approximately equal. 100% of students in these schools qualify for free or reduced lunch.
Intervention: The proposed intervention, DSCoVAR, will be a web-based ITS, which will expose learners to new words in a variety of well-controlled contexts. It will use computational methods to estimate changes in partial word knowledge and will provide strategy training to promote effective use of context cues to meaning.
Research Design and Methods: In Year 1, the research team will develop a fully functioning version of the DSCoVAR system, which entails development and testing of stimuli, tasks, and algorithms that are key components of the system. In Year 2, the research team will conduct iterative usability studies to see if intended end users can understand and use the system. Participants in these studies will fill out a pre-survey, interact with the ITS, and then fill out a post-survey. Additionally, the research team will conduct feasibility studies to see if end users can feasibly implement the intervention in authentic education delivery settings. The feasibility studies will have the same design and procedures as the pilot study. Year 3 will be devoted to a pilot study evaluating the intervention using a randomized controlled trial design. Participants within each grade will be randomly assigned to the control or to one of two intervention groups – the ITS with enhanced, explicit training on the use of context cues to acquire meaning, or the ITS with no enhanced training – to see if explicit training will lead to better word learning outcomes. All participants will be given a pre-test, an immediate post-test, a 1-week delayed post-test, and a 1-month delayed post-test.
Control Condition: In the control condition, participants will receive “business-as-usual” vocabulary instruction.
Key Measures: During iterative development, the research team will collect information about the usability of the system with surveys that ask participants to rate their experiences with various aspects of the system. For the pilot study, the research team will collect both proximal and distal measures of vocabulary knowledge. The proximal measure will be a researcher-designed test that captures students’ knowledge of the Tier 2 words targeted in the intervention. The distal measures will be the vocabulary and passage-level comprehension subscales of the Group Reading Assessment and Diagnostic Evaluation (GRADE) battery. In addition, the research team will develop measures to collect data on two aspects of fidelity of implementation: percent of completed trials and adherence.
Data Analytic Strategy: For all data analyses, the research team will apply one- or two-level generalized linear models (depending on nesting scenario) to assess the relationships between a given outcome measure and covariates. This will include the experimental manipulation and individual difference factors (pre-test vocabulary and comprehension scores and demographic variables such as age, gender, grade level, and socio-economic status).
Frishkoff, G., Collins-Thompson, K., Nam, S.J., Hodges, L., and Crossley, S. (2017). Dynamic Support of Contextual Vocabulary Acquisition for Reading (DSCoVAR): An Intelligent Tutoring System for Contextual Word Learning. Adaptive Educational Technologies for Literacy Instruction. Taylor & Francis eBooks.
Journal article, monograph, or newsletter
Frishkoff, G.A., Collins-Thompson, K., Hodges, L., and Crossley, S. (2016). Accuracy Feedback Improves Word Learning From Context: Evidence From a Meaning-Generation Task. Reading and Writing, 29(4): 609–632.
Nam, S.J., Collins-Thompson, K., and Frishkoff, G. (2016). Modeling Off-Task Behaviors in a Meaning Generation Task. A4L Learning Analytics Monograph. Measurement in Digital Environments paper series, SRI International.
Nam, S.J., Frishkoff, G., and Collins-Thompson, K. (in press). Predicting Off-task Behaviors in an Online Meaning-Generation Task. IEEE Journal on Learning Technologies.
Syed, R. and Collins-Thompson, K. (2017). Optimizing Search Results for Human Learning Goals. Information Retrieval Journal: 1–18.
Kalgren, J., Callin, J., Collins-Thompson, K., Gyllensten, A.C., Ekgren, A., Jurgens, D., Korhonen, A., Olsson, F., Sahlgren, M., and Schutze, H. (2015). Not in the Proceedings of CLEF 2015 TOC. It doesn't come up in a Google search either. Are the details correct? SMA.
Nam, S.J. (2016). Predicting Off-Task Behaviors in an Adaptive Vocabulary Learning System. In 9th International Conference on Educational Data Mining (pp. 672–674). Raleigh, NC: International Educational Data Mining Society (IEDMS).
Syed, R. and Collins-Thompson, K. (2016). Optimizing Search Results for Educational Goals: Incorporating Keyword Density as a Retrieval Objective. In SIGIR 2016 Workshop on Searching as Learning (SAL 2016) (pp. 1–5). Pisa, IT.