Purpose: Too many students fail to develop essential comprehension skills necessary to be successful in school and in their future endeavors. Improving literacy skills relies on the availability of good assessments that can be used to target instruction. The goal of this project is to further develop a comprehensive computerized test battery that assesses the ability of middle and high school students in processing all levels of written and spoken language. Development of the battery for high school students began in a 2005 IES grant http://ies.ed.gov/funding/grantsearch/details.asp?ID=171.

Project Activities: The battery will be field-tested with a large number of middle and high school students with the end goal being a fully-developed battery that has good validity and reliability, is developmentally sensitive, and can differentiate good and poor comprehenders in both reading and listening domains. Instructional materials will be developed for teachers articulating how results can be used to improve student performance in English Language Arts.

Products: The expected outcome from this grant will be an assessment battery for students in grades 6 – 10 as well as a technical manual, instructional materials for teachers, and published reports.

Structured Abstract

Setting: Assessments will be developed at Boston University and field-tested in the East Syracuse-Minoa School district.

Population: A racially and economically diverse group of middle and high school students.

Intervention: The assessment battery will assess all levels of language processing – sub-lexical, lexical, morphological, sentential, and discourse levels. Materials will assess specific features of language, and separate subtests will assess the ability to recognize the forms of language and to understand the contribution of each level of language to overall meaning. All tests will measures reaction time for individual items as well as accuracy. The battery will be administered and scored on a web-based platform.

Research Design and Methods: The refinement of the assessment battery in this project will include the following activities: (1) further assessment of the criterion validity; (2) improving the reliability of some subtests; (3) development of an alternate form; (4) development of a web-based platform; (5) development of a computerized scoring system; and (6) creation of web-based instructional materials for teachers. Alternate form reliability will be assessed by administering both forms of the battery to a sample of students. Test-retest reliability will be assessed by administering the assessment to a sample of students six weeks after the initial assessment. Criterion validity will be assessed through comparison of performance on the battery with that on other standardized tests. Construct validity will be examined by studying the correlations among subtests.

Control Condition: There is no control condition.

Key Measures: The assessment battery includes measures of accuracy and reaction time for items testing the ability to recognize and understand specific aspects of language. Student performance on the battery will be compared with performance on the Woodcock Word Attack and Word Identification subtests, the Peabody Picture Vocabulary Test, the Comprehensive Assessment of Spoken Language sentence comprehension test, the Test of Adult and Adolescent Language 3 written grammatical comprehension, the listening comprehension subtest of the Wechsler Individual Achievement Test, and the em>Ravens Progressive Matrices Test.

Data Analytic Strategy: Measures of reliability (e.g., Cronbach’s alpha, split-half reliability), criterion validity (Pearson r), individual item value (point bi-serial correlation), effect of manipulated variables and grade (ANOVA), hidden structure of performance (exploratory and confirmatory factor analysis; structural equation modeling), and predictive value for real-world performance (multiple and stepwise regression) will be used to support the refinement of this battery.

Related IES Projects: Assessment of Comprehension Skills in Older Struggling Readers (R305G050083)

Products and Publications

Journal article, monograph, or newsletter

Caplan, D., Waters, C.G., Bertram, J., Ostrowski, A., and Michaud, J. (2016). Effects of Written and Auditory Language Processing Skills on Written Passage Comprehension in Middle and High School Students. Reading Research Quarterly, 51(1): 67–92.

** This project was submitted to and funded under Interventions for Struggling Adolescent and Adult Readers and Writers in FY 2010.