Co-Principal Investigators: Kathy Hirsh-Pasek (Temple University) and Jill de Villiers (Smith College)

Purpose: Language is a core ability that children need for success in school. Understanding teachers and peers, following narratives, telling stories, participating in conversation, learning to read, and doing math all rest on linguistic skill. The purpose of this project is to develop a reliable, valid, norm-ready, research-driven, and culturally sensitive computer-based language assessment for children 3- to 5-years-old that can be administered in 20 minutes. The preschool language assessment tool is intended to be an easily administered, automatically scored, 48- item tool that is appropriate for use by teachers, paraprofessionals, and professionals. The tool will have the capacity to quickly and automatically derive individual and group language profiles in two areas of competency: (a) vocabulary and word learning strategies, and (b) grammar and the use of syntax in comprehension. These competencies comprise a broad profile of verbal abilities that reflect both the products of learning, or milestones, and learning processes or strategies. They are predictive of later academic success.

Project Activities: To achieve the goals of this project, researchers will follow a systematic test development process. In the initial year of the project, the team will generate 96 items, create the computer administration and scoring process system, pilot test the items, and complete validity confirmation. In year 2, the team will carry out the first item tryout and will complete initial item analyses in order to reduce the number of items to be administered in the full test. Year 3, activities will include ongoing item analyses and item revisions as results from year 2 activities are completed. Researchers will also conduct psychometric analysis of data collected during the first item tryout phase. In the final year of the project, the team will carry out a second field test using the refined (and smaller) set of items, and will conduct a final set of psychometric analyses. As researchers design the English version, they will also create and pilot test a Spanish version of the assessment. This initial data will prepare researchers to validate the Spanish version in a subsequent project.

Products: Products include a fully developed and validated preschool computer-based language assessment and published reports of research findings.

Structured Abstract

Setting: This study will be conducted in preschools, day care settings, and Head Start classrooms on the eastern seaboard.

Population: Participants include over 1,000 children across four years and includes a mix of African-American, Asian, Hispanic, and Caucasian children. The sample will be balanced for gender and diverse in socio-economic status. Each site will test 100 children (total 300) so that some geographic variation is represented as well as urban, rural, and suburban samples. In year 1, they will work with an initial sample of 60 children (20 in each of three age ranges). In year 2, they will administer the initial item pool to 300 children (100 at each research site). In years 3 and 4, they will work with at least 360 children each year.

Measure: This 20-minute, easy-to-use and scalable preschool language assessment tool is expected to help teachers and providers become sensitized to what children do and do not know about language and to understand the importance of language growth. Assessing crucial aspects of children's linguistic knowledge, the final technologically-delivered 48-item tool is intended to chart normal development, to pinpoint when and where development goes awry, and to use that information to create interventions. The assessment will offer individual and group data. Individual data generated from the use of this system may be used to inform research about the typical course of language development and is intended to provide information that can function as a "red flag" for children with slower developing language. Group data is intended be useful as a research tool and as a way to compare curricular and language interventions.

Research Design and Methods: Test development will proceed in four phases. In Year 1, item generation, pilot testing, and validity confirmation will be completed. Researchers will generate twice the number of items (96) needed on the final test. Content and face validity will be assessed and pilot testing will assist in item adjustment. At the same time, the team will develop the computer system to support the administration and scoring of the items. In Year 2, first item tryout and initial item analysis will occur. The purpose of the first item tryout will be to evaluate a sufficient number of items, and to ensure that the three subscales being assessed: vocabulary, grammar, and process are measured with the final selection of items. More specifically, the first item tryout has the goal of reducing 96 items to the 60 best items. The item tryout will be carried out in each of three age ranges. The project team will also begin item analyses that continue in Year 3 as items are revised. In Year 3, researchers will conduct psychometric analyses using the data from the Year 2 First Item Tryout. The team will also prepare the test items for the Second Tryout Test in Year 4. In Year 4, they will conduct a second field test and conduct a final set of psychometric analyses. Also In Year 4, field tests are continued and detailed and rigorous item evaluations conducted in Year 2 and are repeated in Year 4.

Control Condition: There is no control condition.

Key Measures: The project team will administer the computer-based language assessment in both English and Spanish (as appropriate). Parents of all children who speak a language other than English at home will be administered a short questionnaire. Convergent validity will be assessed using two established standardized language tests: the Peabody Picture Vocabulary Test, Fourth Edition (PPVT-IV), a norm-referenced assessment for the evaluation of receptive vocabulary in children as young as 2 years 6 months, and the Preschool Language Scale Fourth Edition (PLS-4), a widely used test with emphasis on inflectional morphology and acquired vocabulary. Divergent validity will be assessed using a non-verbal spatial test, the Kaufman's Triangles Test. The Triangles subtest is part of the Kaufman Assessment Battery for Children, second edition (KABC-2) and measures children's visual-spatial ability to physically rotate, translate, and combine shapes to form a single compound shape.

Data Analytic Strategy: Reliability and validity will be evaluated throughout the development process. The research team will conduct an exploratory factor analysis to evaluate whether the test items will load appropriately on each subscale of the measure. Rasch analyses and differential item functioning analyses will be conducted after each field test. Additional analyses will be conducted to examine the internal consistency of the test items.

Products and Publications

Book chapter

de Villiers, J. (2015, May). Taking Account of Both Languages in the Assessment of Dual Language Learners. In Seminars in Speech and Language (Vol. 36, No. 02, pp. 120–132). Thieme Medical Publishers.

Journal article, monograph, or newsletter

Levine, D., Strother-Garcia, K., Golinkoff, R. M., and Hirsh-Pasek, K. (2016). Language Development in the First Year of Life: What Deaf Children Might be Missing Before Cochlear Implantation. Otology & Neurotology, 37(2), e56–e62