Project Activities
This project will use a variety of assessment methods to develop and validate a computerized adaptive testing assessment, CASA, to assess early language and literacy skills in low performing, high-risk children who speak English or Spanish. The project will also estimate the costs of using the assessment through a cost analysis.
Structured Abstract
Setting
This research will take place in preschool classrooms across various regions of the country, including Oregon, California, Utah, Minnesota, and Puerto Rico.
Sample
This project will include 800 3- to 5-year-old children who are at risk for later reading difficulties, including (a) children with developmental language disorders, (b) children whose first language is Spanish (monolingual or bilingual), and (c) children identified as at risk through individual performance or demographic factors. To inform the project at various stages, the study will also include up to 80 teachers, a panel of five experts (in preschool language development, speech and language disorders, Spanish language development, and early literacy skill development), and five preschool administrators.
This project will develop CASA, an engaging and instructionally relevant progress monitoring storybook assessment delivered via tablet application for preschool children who have low language and early literacy skills and are at risk for later reading disabilities and challenges. All items will be scaled for use in a computer-adaptive testing engine so that for each scene in the story, children interact with different items based on their current ability level. The assessment is expected to take approximately 15 minutes and measure three language domains (oral language, phonological awareness, and alphabet knowledge) in English and Spanish. CASA will provide rigorous, actionable information for teachers to promote language and early literacy skills for their high-risk students who speak English or Spanish.
Research design and methods
This study will proceed in five phases that iteratively engage an agile application development process to develop CASA. In the first phase, the research team will develop story content in English and Spanish and assessment items that will be integrated into the storybook. In the second phase, a prototype of the assessment will be iteratively developed and tested. A field test with teachers and children in preschool settings will be used in the third phase to test how the items work and merge into constructs (factor structure) and the full suite of items in Spanish and English will be developed. In the fourth phase, items will be integrated into the computer-adaptive administration platform. In the final phase, the application will be revised to incorporate feedback from previous stages to produce a full computer-adaptive storybook experience for children with complementary teacher administration manuals and supports. During this phase, CASA will be used across the academic year to collect data to model growth and examine validity against criterion measures administered at the beginning and end of the academic year.
Control condition
Due to the nature of the research design, there is no control condition.
Key measures
To validate CASA, the research team will use a variety of standardized, norm-referenced criterion measures, including the Individual Growth and Development Indicators in English and Spanish, the Quick Interactive Language Screener in English and Spanish, the MacArthur Bates-Communicative Development Inventories III and MacArthur Inventarios del Desarrollo de Hablidades Communicatives, and the Bilingual Assessment of Phonological Sensitivity. There will also be investigator-developed measures to understand implementation, including focus group feedback forms, fidelity of implementation checklists, and feasibility and utility surveys. Finally, the team will collect a variety of data from the families of children who participate and their teachers using a family survey, the Language Environment Evaluation Report, a classroom survey, and teacher demographic data.
Data analytic strategy
Rasch modeling will be used to scale item difficulties. The team will also conduct confirmatory factor analysis to evaluate how well items measure a single underlying construct for each domain, Pearson's r to correlate CASA scores with criterion measures to test for validity, and HLM to model growth and assess sensitivity to growth.
Cost analysis strategy
Once CASA is completed, the ingredients method will be used to collect data for a cost analysis. The analysis will be conducted using the CostOut tool to estimate individual costs. Using this information, the team will prepare three scenarios that examine costs related to the product, materials, assessor training, school time, teacher or assessor time, and other factors.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: The primary product of this project is a fully developed, validated, and functional app that administers computer-adaptive testing to monitor progress in early language and literacy for English- or Spanish-speaking preschool children, including those who have disabilities, are dual-language learners, or are otherwise at risk for later reading failure. The project will result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders, such as practitioners and policymakers.
ERIC Citations: Find available citations in ERIC for this award here.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
