Project Activities
Researchers will develop and validate Hmong-IGDIs using well-developed empirically sound procedures to produce Hmong screening measures that are psychometrically robust, easy to use, and appropriately sensitive (with cut scores) for identifying students who are ideal candidates for language and early literacy intervention. They will examine the degree to which Hmong-IGDIs cut scores and student performance predicts kindergarten and 1st grade English early reading outcomes.
Structured Abstract
Setting
Project activities will take place in early childhood programs in the west (California) and the midwest (Minnesota and Wisconsin) United States. This will allow the research team to recruit students with diverse samples of dialects and regional representation.
Sample
The sample includes early childhood educators and 4- and 5-year-old typically developing Hmong dual language learners. A total of 760 Hmong children will be recruited and participate in 5 of the 6 Phases of this project. A panel of at least 10 Hmong teachers and/or educational professionals in early childhood will serve as an ongoing review panel.
Assessment
The Hmong-IGDI measures will be brief, easy to use tools designed as general outcome measures to assess domain- general skills in Hmong language and early literacy. The measures will be designed to take 1-3 minutes to administer and function as screening tools to appropriately identify 3 to 5-year-olds who may benefit from additional intervention. Researchers will also develop supporting tools and resources, including a comprehensive training package, a comparative score rubric to evaluate scores and links to existing intervention resources, within a database decision making model.
Research design and methods
The research team will develop Hmong-IGDIs over a 4-year period. Researchers will design the measures using Kane's validity argument framework, Wilson's measure design methodology, Rasch modeling and other strong psychometric methods to limit bias and construct-irrelevant features during the design and validation process. The project activities will be completed in six phases. In Phase 1, researchers will design and field test Hmong-IGDIs measures and items. In Phase 2, researchers will expand the successful candidate Hmong-IGDIs measures to 75 items and then calibrate these items with a larger, more representative sample. They will collect item level responses from 3- to 5-year-old Hmong preschoolers using Hmong-IGDIs at seasonal time points (fall, winter & spring) and then use resulting data to establish Rasch item locations and describe performance over time. In Phase 3, researchers will use the contrasting groups standard setting process to set screening cut scores. They will use teacher and parent judgments of child performance to identify Hmong-IGDIs scores and set the cut scores. In Phase 4, researchers will develop two resources: (1) a full manual, including training protocols, technical qualifications, standardization, and frequently asked questions and (2) a Comparison Score Rubric (CSR). The CSR will be used to examine base rates by intervention candidacy in Hmong and English (on IGDIs) and to produce instructional recommendations including Hmong intervention led by development work in the SPPS Hmong immersion PreK programs. In Phase 5, researchers will conduct a predictive validity study. They will examine how performance on IGDIs in Hmong and English predict status and growth in kindergarten for a cohort of children. They will use the FASTBridge Assessment Peabody Picture Vocabulary Test, and the Hmong Language Assessment to assess predictive relations in English and Hmong. In Phase 6, researchers will engage in rapid iterative feedback cycles with their community teacher panel. Teachers will contribute to Hmong-IGDIs design at every phase and provide direct input on feasibility, content design and capture capacity to reach fidelity on standardized measure protocols.
Key measures
Across the four years of this project, measure development will focus on Hmong-IGDIs item development and item level statistics including Rasch difficulty, standard error of measurement, infit, outfit, p-value, and point biserial correlations. Additional criterion measures include the measures given in kindergarten and 1st grade: Hmong Language Assessment, FASTBridge Assessment and the Peabody Picture Vocabulary Test.
Data analytic strategy
The research team will use a variety of measurement analyses to calibrate and scale the Hmong-IGDIs. Methods include Rasch modeling for item calibration, differential item functioning via the Rasch-Welch statistic to evaluate bias in the item content, confirmatory factor analysis to confirm unitary construct representation, Pearson's r to correlate Hmong-IGDIs with English criterion measures in kindergarten, Receiver Operator Curve analyses to examine sensitivity and specificity and contrasting groups design analyses to set cut scores and Regression to examine to what degree Hmong-IGDIs predict kindergarten and 1st grade English and Hmong reading outcomes.
Cost analysis strategy
Once Hmong-IGDIs are completed, researchers will use the ingredients method to compute cost of the assessment through cost analysis procedures. They will use the CostOut tool to estimate individual costs (www.cbcsecosttoolkit.org). Researchers will use this information to prepare three scenarios that examine costs related to the product, materials, assessor training, school time, teacher or assessor time and other factors.
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project will include the Hmong-IGDIs assessment suite for screening Hmong language and early literacy development in 3-5-year-old children. Once completed, the measures will have immediately utility in three areas of the nation with densely populated Hmong communities: St Paul, MN; Madison, WI and Fresno, CA. The researchers will disseminate findings from this study through conference presentations, peer-reviewed publications, and presentations and online resources for practitioners.
Supplemental information
Co-Principal Investigators: Erickson, Lori; Rodriguez, Michael; Vang, Gao
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
