|Title:||Uno, Dos, Tres, Listos! Monitoring Kindergarten Readiness Bilingually|
|Principal Investigator:||Grigorenko, Elena L.||Awardee:||University of Houston|
|Program:||Education Technology [Program Details]|
|Award Period:||4 years (9/2/2016 – 8/31/2020)||Award Amount:||$1,399,252|
Previous Award Number: R305A160233
Purpose: Researchers in this project are developing and validating a classroom-based, bilingual (Spanish/English) adaptive assessment of children's readiness for knowledge and skills aligned to Early Learning and Development Standards. Due to varying experiences with how children learn a second language in and outside of the home, English learners often present a range of linguistic profiles. These children's academic achievement may be slowed as they are simultaneously developing language and skills in both Spanish and English. An additional obstacle for English learners is a lack of appropriate and accurate assessments, as there remains a dearth of well-validated assessments for teachers to appraise children's skill levels and learning over time, and tailor instruction based on this feedback.
Project Activities: The researchers will field-test a set of paper-based assessment items to calibrate children's responses and inform the development of the adaptive algorithm of the assessment. They will develop prototypes of the assessment, while collecting data at multiple points to inform iterative refinements. They will evaluate the psychometric properties of the items in a controlled setting and validate the fully functional computerized adaptive assessment with children in classrooms.
Products: Researchers will develop a touch-screen tablet-based assessment for Spanish-speaking English learners. The assessment will provide systematic feedback and scaffolding for incorrect and incomplete responses. A teacher manual will accompany the assessment.
Setting: Development and validation of the assessment will occur in two medium-sized suburban districts in Connecticut (30% are Hispanic) and Texas (55% are Hispanic).
Sample: The sample will consist of 2,800 children 3 to 5 years of age. The researchers will employ a two-stage sampling strategy in Years 1 and 2, with a non-random sample participating in studies to develop and refine the assessment. In Year 3, researchers will construct a sample using stratified random sampling based on the entire preschool population from both districts. Children who participated in the pilot study in Year 1 and 2 will be excluded from participation afterward, as these children will have previously encountered a considerable number of items that were initially administered in field tests.
Assessment: The BELLA (Bilingual English Language Learner Assessment) will be a touch-screen tablet-based assessment for Spanish-speaking English learners. Researchers will administer the assessment in both English and Spanish, with items initially presented to English Learners in Spanish to determine knowledge and skill level in the primary language, and then in English. The adaptive engine will provide systematic feedback and scaffolding for incorrect and incomplete responses. The assessment will measure children's cognitive skills for analytical, creative, and practical thinking, and knowledge in aspects of early literacy, numeracy, and science.
Research Design and Methods: In Year 1 and 2, researchers will develop and field-test a set of paper-based assessment items, and mock-ups of artwork, animations, and plans for administering the assessment. During this stage, the research team will calibrate children's responses to inform the adaptive algorithm of the assessment. Next, the team will develop prototypes of the assessment, while collecting data from children at multiple points to inform iterative refinements. In Year 3, researchers will evaluate the psychometric properties of the items and the assessment in a controlled setting and update the items features as needed. In Year 4, researchers will conduct studies to validate the assessment with a new cohort of children in classrooms. The researchers will employ a mix of qualitative and quantitative methods throughout the course of the project.
Control Condition: There is no control condition for this project.
Key Measures: Researchers will develop key measures and will focus on children's proficiency in kindergarten readiness in areas of literacy, numeracy, and science, along with measures of cognitive skills by which knowledge is acquired or mastered (via analytical skills), extended or transformed (via creative skills), and utilized or applied (via practical skills).
Data Analytic Strategy: The research team will use item response analyses to test the underlying structure of BELLA and to provide parameters for adaptive item selection and the scoring algorithm. The validation stage will examine the construct and predictive validity of BELLA at pre-school entry and before kindergarten entry. To examine construct validity the research will relate proficiency levels generated by BELLA to established performance assessments. To examine predictive validity, researchers will track children a minimum of three time points to analyze change over time.