Project Activities
The goal of this project is to further develop, validate, and disseminate the CBAS-R. The project team will work with practicing teachers to develop a bank of about 2,600 items. Items for the bank will be designed for both skills analysis and progress monitoring, as well as to assess both broad and component reading skills. Items will be field tested with K–5 students in mobile computer labs, using a matrix sampling approach which will ensure each item is administered to about 500 students but no student will need to complete all 2,600 items. This item bank forms the foundation of the computer-adaptive measure. Individual students will typically take a 20-item adaptive test that is anticipated to take eight to twelve minutes to administer. Feedback collected from teachers and students will inform the validity and feasibility of the CBAS-R. The assessment will be disseminated via the CBAS Application Software System for web-based administration, thus providing the CBAS-R to teachers throughout the United States at low or no cost.
Structured Abstract
Setting
School districts in rural, suburban, and urban Minnesota will participate in the project.
Sample
Participants will include approximately 13,000 students in grades K through 5.
Intervention
Researchers will develop and validate the CBAS-R, a computer-adaptive reading instrument aimed at skills assessment and progress monitoring for use with students in grades K–5. The CBAS-R has already been developed as a tri-annual assessment of broad reading skills in grades K–5. However, the new CBAS-R will assess both broad and component reading skills including: concepts of print; phonological skills; automaticity/fluency; phonics; vocabulary; and comprehension. Assessment score reports will be automatically generated for teachers to aid in identifying students' strengths and weaknesses, monitor the progress of students' reading skills, and inform instructional choices. The project team anticipates a 20-item computer-adaptive measure that will take eight to twelve minutes to administer. Previous feasibility work with the CBAS-R indicates that teachers believed the assessment was not disruptive to classroom activities and that it was beneficial to instruction.
Research design and methods
The CBAS-R will be developed in four phases as follows: Phase (1) item development, parameterization, and modeling; (2) establish procedures for administrations aimed at skills analysis and progress monitoring; (3) evaluate the validity and feasibility of the CBAS-R; and (4) disseminate the CBAS-R materials and research. The project team will work with a group of teachers to develop the items. A panel of experts will review and evaluate the items in order to optimize accuracy, content relevance, fairness, and difficulty. The expert panel will also conduct a sensitivity review to ensure the items are appropriate for youth from all racial, ethnic, and gender groups. Following item development and revisions, the items will be field tested with a group of approximately 11,250 students in grades K–5. Phase 2 will involve the development of administration procedures for the skills analysis and progress monitoring features of the CBAS-R. This process will enable the project team to choose the items that provide the most information regarding students' strengths and weaknesses to include in the final version. Additionally, items will be balanced for content, and criteria for terminating the assessment will be developed. Phase 3 of the project will involve validity and feasibility studies. Validity will be assessed psychometrically, including examining the correlations between CBAS-R scores and other measures of reading achievement. Feasibility studies will include assessing both the ease of test administration, and teachers' ease of interpreting and using the results of the CBAS-R. The final phase, Phase 4, will involve the final development and dissemination of the CBAS administration software.
Control condition
Due to the nature of the research design, there is no control condition.
Key measures
Criterion-related validity of the CBAS-R will be assessed using students' reading scores from the Minnesota State reading test, the Measures of Academic Progress, the Curriculum-Based Measurement of Reading, and the Dynamic Indicators of Basic Early Literacy Skills. Interviews, rating scales, and observations will be used and allow the project team to assess feasibility of the assessment in classrooms.
Data analytic strategy
Each of the first three phases of the project will involve different data analytic strategies. Phase 1 will use a three-parameter Item Response Theory (IRT) design to model the parameters of item difficulty, item discrimination, and an estimate for guessing. Additionally, the project team will use a bifactor model (which assumes each item loads on a general factor and only one component factor) and full-metric concurrent calibration, so that the final CBAS-R will produce estimates for broad and component reading skills on the same scale. Phase 2 will include a Differential Item Functioning analysis which will allow the project to determine if the CBAS-R items function the same for members of various racial, ethnic, and gender groups. Finally, Phase 3 will include tests of validity and reliability using IRT and testing for correlations with reading scores from a variety of assessments already in use.
People and institutions involved
IES program contact(s)
Products and publications
Products: The product of this project will be a fully developed and validated computer-adaptive assessment of reading skills in grades K-5 entitled the Computer-Based Assessment System for Reading (CBAS-R). Peer reviewed publications will also be produced.
Journal article, monograph, or newsletter
Kendeou, P., McMaster, K.L., Christ, T.J. (2016). Reading Comprehension: Core Components and Processes. Policy Insights from the Behavioral and Brain Sciences, 3, 62-69.
Supplemental information
Co-Principal Investigators: Barbara Taylor and David J. Weiss
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
