|Title:||National Accessible Reading Assessment Projects: Research and Development for Students with Visual Impairments|
|Principal Investigator:||Laitusis, Cara Cahalan||Awardee:||Educational Testing Service (ETS)|
|Program:||Systems, Policy, and Finance [Program Details]|
|Award Period:||7/1/06 to 6/30/2011||Award Amount:||$1,992,629|
Funded under the Assessment for Accountability topic prior to the establishment of the Systemic Interventions and Policies for Special Education topic.
Purpose: Students with visual impairments present an ongoing challenge in large-scale assessments of reading proficiency, and issues persist concerning the valid assessment of reading with this population, particularly when some type of technology-assisted reading (e.g. text to speech) is involved. The purpose of this project is to: (a) examine the psychometric properties of state assessments for students with visual impairments, (b) conduct research on the development of an alternate assessment of technology-assisted reading for students with visual impairments, and (c) conduct a field test to examine the validity and reliability of both an accessible reading assessment and the alternate assessment of technology-assisted reading that includes students with visual impairments.
Project Activities: The researchers will address the above research goals to develop and validate measurement tools through several strategies. First, the research team will analyze data from a state testing program to examine the psychometric properties of tests administered to students with visual impairments. In addition, supplemental analyses will be employed to compare subgroups by accommodation. Finally, a field trial will be conducted of an accessible assessment of reading comprehension and the technology-assisted reading prototype for students with visual impairments. Analysis will examine the degree to which both prototype assessments add significant amounts of information to current state reading assessments for students with visual impairments.
Products: The expected outcomes from this study include:
Setting: Research on psychometric properties will be conducted on data from the California Standardized Testing and Reporting (STAR) assessment, a large statewide assessment program. A survey of users of technology-assisted reading and their teachers will employ a national sample and will be conducted via the world wide web. The development and testing of the assessment instruments will take place in 8th grade settings, with specific locations as yet to be determined.
Population: This project focuses on students who are blind or visually impaired. The analysis of California STAR data includes 4th and 8th grade students who are identified as having visual impairments, some of whom used accommodations on the assessment. A sample of students without disabilities is included as a reference group. The survey of users of technology-assisted reading and their teachers includes students from grades 7 through 9 and their teachers. The development and testing of the assessment instruments will include 8th grade students with visual impairments. Specific sample sizes are not yet determined.
Intervention: The intervention of primary interest is an assessment of technology-assisted reading proficiency. The project will also study a test of reading proficiency that is being designed under separate funding to be accessible for students with disabilities.
Research Design and Methods: Data from the California STAR assessment will be analyzed to determine psychometric properties related to students with visual disabilities. The survey of users of technology-assisted reading and their teachers will be administered via the World Wide Web to identify technology platforms, approaches to accessing written material, reading constructs, etc. The field test of the assessment instrument will employ a counter-balanced within-subjects experimental design including students with and without visual impairments receiving assessments under varying conditions.
Control Condition: Assessments without accommodations, assessments without accessibility features, and students without disabilities will all serve as counterfactuals in the various studies conducted by this project.
Key Measures: Key measures include the English and Language Arts assessment of the California Standardized Testing and Reporting (STAR) program, the assessment of technology-assisted reading to be developed and tested by this project, and the accessible reading assessment instrument to be developed under separate funding.
Data Analytic Strategy: Data from the California STAR assessment will be analyzed for reliability, score distributions, and completion rates for various groups. These data will also be examined for differential item functioning (DIF) for different groups. Data from the field test of assessment instruments will be analyzed by means of multiple regression analysis, and if numbers are sufficient, DIF analysis.
* The dollar amount includes funds from the Office of Special Education Programs (OSEP) and NCSER.
Journal article, monograph, or newsletter
Stone, E., Cook, L., Cline, F., and Cahalan-Laitusis, C. (2010). Using Differential Item Functioning to Investigate the Impact of Testing Accommodations on an English-Language Arts Assessment for Students who are Blind or Visually Impaired. Applied Measurement in Education, 23(2): 132–152.
Nongovernment report, issue brief, or practice guide
Johnstone, C., Altman, J., Timmons, J., and Thurlow, M. (2009). Students With Visual Impairments and Assistive Technology: Results From a Cognitive Interview Study in Five States. Minneapolis, MN: University of Minnesota, Technology Assisted Reading Assessment.
Johnstone, C., Altman, J., Timmons, J., Thurlow, M., and Laitusis, C. (2009). Field-Based Perspectives on Technology Assisted Reading Assessments: Results of an Interview Study With Teachers of Students With Visual Impairments (TVIs). Minneapolis, MN: University of MN, Technology Assisted Reading Assessment.
Thurlow, M., Johnstone, C., Timmons, J., and Altman, J. (2009). Survey of Teachers of Students With Visual Impairments: Students Served and Their Access to State Assessments of Reading. Minneapolis, MN: University of MN, Technology Assisted Reading Assessment.