Description:

Co-Principal Investigators: Charles Perfetti (University of Pittsburgh), Jack Mostow (Carnegie Mellon University)

Purpose: Vocabulary knowledge is an important component of academic success and impacts skills across academic disciplines. It is a critical component of literacy development. However, the vocabulary skills of high school students in the U.S. are insufficient to meet the demands of college and career. Furthermore, there is very little vocabulary instruction taught in U.S. high schools. The current project will develop a high school vocabulary intervention that builds on the fully developed DictionarySquared vocabulary tutor. The current version of DictionarySquared teaches vocabulary by exposing students to words in multiple authentic contexts. Through iterative development, the new version of DictionarySquared will be designed to adapt to student knowledge, use high-quality contexts, encourage active processing and student engagement, and assess student learning with reliable pre- and post-tests.

Project Activities: During Years 1 and 2, the research team will iteratively develop the new version of DictionarySquared. The iterative development phase will include implementation, observation, and revision to the intervention. A pilot study will be conducted in Year 3 with three groups: a new DictionarySquared treatment group, an existing "baseline" DictionarySquared control group, and a business-as-usual control group.

Products: The final product of this project will be the revised, fully developed DictionarySquared web-based intervention for high school students. Peer-reviewed publications will also be produced.

Structured Abstract

Setting: This study will be conducted in high schools in California, Ohio, Pennsylvania, South Carolina, and Texas.

Sample: Approximately 750 to 870 high school students will participate in this study. For the iterative development phase of the project, 450 students in 9th through 12th grade will participate in the study. Approximately 300 to 420 students in 9th through 12th grade will participate in the pilot study.

Intervention: The final version of this intervention will be a web-based intervention aimed at increasing high school students' vocabulary learning. The web-based platform will include five new components: (1) assessments of word knowledge; (2) adaptive review schedule; (3) informative contexts for vocabulary instruction; (4) active processing activities for students; and (5) engaging features to encourage students to use the intervention. The intervention is designed to be integrated into language arts curriculum for 10-15 minutes a day.

Research Design and Methods: During Years 1 and 2, researchers will design, implement, and revise the five new components of DictionarySquared. The assessments of word knowledge will be developed and validated. The adaptive reviews of words will be developed using an embedded within-subjects experiment in which some words are taught under a standard schedule and some under an adaptive schedule. Researchers will develop scalable methods for selecting informative contexts by choosing contexts harvested from the web, having the contexts judged by investigators on the team, and seeking amateur ratings of the quality of the contexts through Amazon's Mechanical Turk. Active processing activities will be developed and tested with students. Engagement features such as games and scoring other students' sentences (i.e., social engagement) will also be developed and tested with students during the iterative development phase. During the final year, 15-21 new classrooms of high school students (approximately 300-420 students) will be recruited and randomly assigned to the treatment group, the baseline version of DictionarySquared group, or the business-as-usual control group.

Control Condition: This study will use two control conditions: one will use the existing "baseline" version of the intervention software; one will be a business-as-usual control.

Key Measures: To examine the efficacy of the intervention to improve students' vocabulary and literacy skills, researchers will administer the researcher-designed vocabulary knowledge pre- and post-test, the Gates MacGinitie Reading Test—4th Edition, the Peabody Picture Vocabulary Test—4th Edition, and the Expressive Vocabulary Test—2nd Edition.

Data Analytic Strategy: Researchers will use a two-level multilevel analysis with students nested within schools to examine the effect of the intervention. Condition (treatment or one of the two controls) will be included in the model as dummy codes. Because there will be three analyses, one for each of the vocabulary outcomes, the researchers will use a Linear Step Up procedure to control for multiple comparisons. This procedure is designed to control the false discovery rate (i.e., finding significant results that do not actually exist) while still maintaining sufficient power. The Linear Step Up procedure ensures that the false discovery rate does not go above 5 percent.

Publications

Journal article, monograph, or newsletter

Adlof, S., Frishkoff, G., Dandy, J., and Perfetti, C. (2016). Effects of Induced Orthographic and Semantic Knowledge on Subsequent Learning: A Test of the Partial Knowledge Hypothesis. Reading and Writing: An Interdisciplinary Journal, 29(3): 475–500.

Proceeding

Mostow, J., Gates, D., Ellison, R., & Goutam, R. (2015). Automatic Identification of Nutritious Contexts for Learning Vocabulary Words. In International Educational Data Mining Society. Madrid, ES: International Educational Data Mining Society.