Co-Principal Investigators: Erickson, Lori; Rodriguez, Michael; Vang, Gao

Purpose: Research on language and literacy has provided information on English development, domain specific skills, and the trajectory of early literacy and language skills. However, little attention has been focused on low incidence populations, including ethnically and linguistically diverse students. Hmong Americans, one subgroup of Asian Americans, represent one of the most underserved cultural communities in the US. Hmong Americans represent an important Asian sub-group that may experience a negative academic and economic trajectory if meaningful intervention is not put into place. To address the early learning needs of Hmong children, this project will develop a high quality, technically sound, and innovative preschool early language and screening tool in Hmong: the Hmong Individual Growth and Development Indicators (Hmong-IGDIs). This project will be carried out by the University of Minnesota in partnership with St. Paul (MN) Public Schools (SPPS) and others, and is based on a line of research and findings from a Research-Practice Partnership project. SPPS requested the development of this new screening tool.  Researchers will work with school district personnel to develop, evaluate, implement, and produce practical application recommendations for assessment of language and early literacy skills for Hmong-speaking preschool children. Hmong-IGDIs will complement existing tools in English (IGDIs). The development of a Hmong screening measure will allow for seasonal assessment to provide meaningful data regarding intervention candidacy, and in turn, will contribute to databased decision making.

Project Activities: Researchers will develop and validate Hmong-IGDIs using well-developed empirically sound procedures to produce Hmong screening measures that are psychometrically robust, easy to use, and appropriately sensitive (with cut scores) for identifying students who are ideal candidates for language and early literacy intervention. They will examine the degree to which Hmong-IGDIs cut scores and student performance predicts kindergarten and 1st grade English early reading outcomes.

Products: The products of this project will include the Hmong-IGDIs assessment suite for screening Hmong language and early literacy development in 3–5-year-old children. Once completed, the measures will have immediately utility in three areas of the nation with densely populated Hmong communities: St Paul, MN; Madison, WI and Fresno, CA. The researchers will disseminate findings from this study through conference presentations, peer-reviewed publications, and presentations and online resources for practitioners.

Structured Abstract

Setting: Project activities will take place in early childhood programs in the west (California) and the midwest (Minnesota and Wisconsin) United States. This will allow the research team to recruit students with diverse samples of dialects and regional representation.

Sample: The sample includes early childhood educators and 4- and 5-year-old typically developing Hmong dual language learners. A total of 760 Hmong children will be recruited and participate in 5 of the 6 Phases of this project. A panel of at least 10 Hmong teachers and/or educational professionals in early childhood will serve as an ongoing review panel.

Assessment: The Hmong-IGDI measures will be brief, easy to use tools designed as general outcome measures to assess domain- general skills in Hmong language and early literacy. The measures will be designed to take 1-3 minutes to administer and function as screening tools to appropriately identify 3 to 5-year-olds who may benefit from additional intervention. Researchers will also develop supporting tools and resources, including a comprehensive training package, a comparative score rubric to evaluate scores and links to existing intervention resources, within a database decision making model.

Research Design and Methods: The research team will develop Hmong-IGDIs over a 4-year period. Researchers will design the measures using Kane's validity argument framework, Wilson's measure design methodology, Rasch modeling and other strong psychometric methods to limit bias and construct-irrelevant features during the design and validation process. The project activities will be completed in six phases. In Phase 1, researchers will design and field test Hmong-IGDIs measures and items. In Phase 2, researchers will expand the successful candidate Hmong-IGDIs measures to 75 items and then calibrate these items with a larger, more representative sample. They will collect item level responses from 3- to 5-year-old Hmong preschoolers using Hmong-IGDIs at seasonal time points (fall, winter & spring) and then use resulting data to establish Rasch item locations and describe performance over time. In Phase 3, researchers will use the contrasting groups standard setting process to set screening cut scores. They will use teacher and parent judgments of child performance to identify Hmong-IGDIs scores and set the cut scores. In Phase 4, researchers will develop two resources: (1) a full manual, including training protocols, technical qualifications, standardization, and frequently asked questions and (2) a Comparison Score Rubric (CSR). The CSR will be used to examine base rates by intervention candidacy in Hmong and English (on IGDIs) and to produce instructional recommendations including Hmong intervention led by development work in the SPPS Hmong immersion PreK programs. In Phase 5, researchers will conduct a predictive validity study. They will examine how performance on IGDIs in Hmong and English predict status and growth in kindergarten for a cohort of children. They will use the FASTBridge Assessment Peabody Picture Vocabulary Test, and the Hmong Language Assessment to assess predictive relations in English and Hmong. In Phase 6, researchers will engage in rapid iterative feedback cycles with their community teacher panel. Teachers will contribute to Hmong-IGDIs design at every phase and provide direct input on feasibility, content design and capture capacity to reach fidelity on standardized measure protocols.

Key Measures: Across the four years of this project, measure development will focus on Hmong-IGDIs item development and item level statistics including Rasch difficulty, standard error of measurement, infit, outfit, p-value, and point biserial correlations. Additional criterion measures include the measures given in kindergarten and 1st grade: Hmong Language Assessment, FASTBridge Assessment and the Peabody Picture Vocabulary Test.

Data Analytic Strategy: The research team will use a variety of measurement analyses to calibrate and scale the Hmong-IGDIs. Methods include Rasch modeling for item calibration, differential item functioning via the Rasch-Welch statistic to evaluate bias in the item content, confirmatory factor analysis to confirm unitary construct representation, Pearson's r to correlate Hmong-IGDIs with English criterion measures in kindergarten, Receiver Operator Curve analyses to examine sensitivity and specificity and contrasting groups design analyses to set cut scores and Regression to examine to what degree Hmong-IGDIs predict kindergarten and 1st grade English and Hmong reading outcomes.

Cost Analysis: Once Hmong-IGDIs are completed, researchers will use the ingredients method to compute cost of the assessment through cost analysis procedures. They will use the CostOut tool to estimate individual costs (www.cbcsecosttoolkit.org). Researchers will use this information to prepare three scenarios that examine costs related to the product, materials, assessor training, school time, teacher or assessor time and other factors.

Related IES Projects: Addressing the Growing Diversity of Preschool Populations through Low Incidence Language Barriers: Hmong Language Development to Improve Assessment Approaches (R305H170073); Expanding Individual Growth and Development Indicators of Language and Early Literacy for Universal Screening in Multi-Tiered Systems of Support with Three-Year-Olds (R305A160034); Making Progress with Progress Monitoring: Developing Early Literacy and Language Individual and Growth Development Indicators as Progress Monitoring Tools (R305A160080)