Project Activities
The research team will develop assessments of vocabulary depth that focus on the measurement of vocabulary learning in content area classrooms. The project team will develop a model of vocabulary breadth, as well as methods to measure vocabulary knowledge systematically at different levels of depth. The team will then use the vocabulary breadth model and the vocabulary depth measures together to create profiles of middle grade student vocabulary knowledge over at least two specific content areas (e.g., world history and middle school biology) within the larger domains of history, geography and biology.
Structured Abstract
Setting
Participating schools are located in a large urban city in Massachusetts.
Sample
Participants include world history and biology students in at least four middle schools, with approximately 120 students per school per grade. Sampling of schools and students will be done such that student diversity in terms of race/ethnicity, socioeconomic status, and English language learner status are well represented.
Research design and methods
The research will fall into three strands. Strand I will focus on modeling the fine topical structure of vocabulary, and on using such a model to define measures of topic-specific vocabulary breadth. Strand II will focus on creating appropriate measures of vocabulary depth. Strand III will bring Strands I and II together in a large-scale study of a middle school student population. In this large field study, assessments of content-area vocabulary that tap both breadth and depth of knowledge will be created using both convergent and discriminant validity.
Key measures
Key measures include researcher-constructed tests with four different forms that test the same item types, the Peabody Picture Vocabulary Test-Revised, and the SARA reading battery. Researchers will also collect demographic data such as native language, course grades, and score on the last state standardized accountability measure.
Data analytic strategy
The researchers will calculate mean accuracy scores for each item type. They expect mean accuracy to vary widely across item types, and to fall into a natural order, reflecting whether the item type is measuring low or high measures of vocabulary depth. An item analysis will also be performed. Finally, the researchers will examine correlations between test scores and external vocabulary and content knowledge measures.
People and institutions involved
IES program contact(s)
Products and publications
Products: The expected outcomes of this research include vocabulary topic maps that provide detailed word lists to guide instruction and assessment in the content areas of history, geography, and biology, a valid item-battery to assess partial vocabulary knowledge, a set of assessments to measure breadth and depth of vocabulary knowledge in two specific domains (world history and biological science) at the middle school level, and published reports describing these outcomes.
Book chapter
Deane, P. (2012). NLP Methods for Supporting Vocabulary Analysis. In J.P. Sabatini, T. O'Reilly, and E. Albro (Eds.), Reaching an Understanding: Innovations in how we View Reading Assessment (pp. 117-144). Lanham, MD: Rowman and Littlefield.
Nongovernment report, issue brief, or practice guide
Deane, P., Lawless, R.R., Li, C., Sabatini, J., Bejar, I.I. and O'Reilly, T. (2014). Creating Vocabulary Item Types That Measure Students' Depth of Semantic Knowledge. Washington, DC: ETS.
Proceeding
Krovetz, R., Deane, P., and Madnani, N. (2011). The Web is not a Person, Berners-Lee is not an Organization, and African-Americans are not Locations: An Anlaysis of the Performance of Named-Entity Recognition. In Proceedings of the ACL 2011 Workshop on Multiword Expressions: From Parsing and Generation to the Real World (MWE 2011) (pp. 57-64). Portland, OR: Association for Computational Linguistics.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
