Project Activities
The researchers will field-test a set of paper-based assessment items to calibrate children's responses and inform the development of the adaptive algorithm of the assessment. They will develop prototypes of the assessment, while collecting data at multiple points to inform iterative refinements. They will evaluate the psychometric properties of the items in a controlled setting and validate the fully functional computerized adaptive assessment with children in classrooms.
Structured Abstract
Setting
Development and validation of the assessment will occur in two medium-sized suburban districts in Connecticut (30% are Hispanic) and Texas (55% are Hispanic).
Sample
The sample will consist of 2,800 children 3 to 5 years of age. The researchers will employ a two-stage sampling strategy in Years 1 and 2, with a non-random sample participating in studies to develop and refine the assessment. In Year 3, researchers will construct a sample using stratified random sampling based on the entire preschool population from both districts. Children who participated in the pilot study in Year 1 and 2 will be excluded from participation afterward, as these children will have previously encountered a considerable number of items that were initially administered in field tests.
Assessment
The BELLA (Bilingual English Language Learner Assessment) will be a touch-screen tablet-based assessment for Spanish-speaking English learners. Researchers will administer the assessment in both English and Spanish, with items initially presented to English Learners in Spanish to determine knowledge and skill level in the primary language, and then in English. The adaptive engine will provide systematic feedback and scaffolding for incorrect and incomplete responses. The assessment will measure children's cognitive skills for analytical, creative, and practical thinking, and knowledge in aspects of early literacy, numeracy, and science.
Research design and methods
In Year 1 and 2, researchers will develop and field-test a set of paper-based assessment items, and mock-ups of artwork, animations, and plans for administering the assessment. During this stage, the research team will calibrate children's responses to inform the adaptive algorithm of the assessment. Next, the team will develop prototypes of the assessment, while collecting data from children at multiple points to inform iterative refinements. In Year 3, researchers will evaluate the psychometric properties of the items and the assessment in a controlled setting and update the items features as needed. In Year 4, researchers will conduct studies to validate the assessment with a new cohort of children in classrooms. The researchers will employ a mix of qualitative and quantitative methods throughout the course of the project.
Control condition
There is no control condition for this project.
Key measures
Researchers will develop key measures and will focus on children's proficiency in kindergarten readiness in areas of literacy, numeracy, and science, along with measures of cognitive skills by which knowledge is acquired or mastered (via analytical skills), extended or transformed (via creative skills), and utilized or applied (via practical skills).
Data analytic strategy
The research team will use item response analyses to test the underlying structure of BELLA and to provide parameters for adaptive item selection and the scoring algorithm. The validation stage will examine the construct and predictive validity of BELLA at pre-school entry and before kindergarten entry. To examine construct validity the research will relate proficiency levels generated by BELLA to established performance assessments. To examine predictive validity, researchers will track children a minimum of three time points to analyze change over time.
People and institutions involved
IES program contact(s)
Products and publications
Products: Researchers will develop a touch-screen tablet-based assessment for Spanish-speaking English learners. The assessment will provide systematic feedback and scaffolding for incorrect and incomplete responses. A teacher manual will accompany the assessment.
Additional project information
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
