Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Development of a Computerized Assessment of Executive Function for Preschool-Aged Children
Center: NCSER Year: 2012
Principal Investigator: Kuhn, Laura Awardee: University of North Carolina, Chapel Hill
Program: Early Intervention and Early Learning      [Program Details]
Award Period: 3/1/2012–2/28/2016 Award Amount: $1,628,302
Type: Measurement Award Number: R324A120033

Co-Principal Investigator: Michael Willoughby

Purpose: Executive function (EF) is an umbrella term that refers to a wide range of cognitive abilities that together serve as a supervisory system important for planning, reasoning ability, and the integration of thought and action. EF plays a central role in children's development of self-regulation and social and cognitive competence. Evidence suggests that EF deficits may act as a "final common pathway" through which diverse disorders and risk factors affecting young children lead to learning difficulties and early school failure. As such, it is essential that psychometrically-sound longitudinal measures of EF be developed to identify EF deficits in diverse groups of children and to determine the efficacy of various attempts at remediating these deficits and promoting school achievement. The aim of this study is to further develop and evaluate a computerized assessment of EF for use with preschool-aged children.

Project Activities: The investigators will complete the development of tasks and computerization of the EF assessment; evaluate the test-retest reliability of the new EF computerized assessment; evaluate the psychometric properties of individual EF tasks and the battery overall; and test the validity of the computerized EF battery by relating task performance to other measures of EF, processing speed and intelligence, academic achievement, and teacher-rated behaviors.

Products: The main product will be a fully developed and validated computerized assessment to identify EF deficits in preschoolers and determine the efficacy of interventions aimed at remediating EF deficits. Data on the reliability, validity, and feasibility of the computer assessment will also be available.

Structured Abstract

Setting: The project will take place in preschools in urban, suburban, and rural settings in North Carolina and New York.

Sample: The sample will include approximately equal numbers of male and female 3- to 5-year-old children reflecting a broad range of skill, stratified by racial and ethnic categories, and proportional to their national representation in the United States Census.

Assessment: The assessment is conducted with a laptop (or netbook) and a standalone touch screen monitor that is connected to the laptop. The computer program displays text on the laptop screen that is read verbatim by interviewers to children describing the nature of the task/item. Simultaneously, the computer program displays test stimuli to children on the touch screen monitor that sits directly in front of them. Children respond to each item by touching the screen. Their response is "captured" using a capacitive touch screen monitor and stored in a comma delimited file on the laptop for later scoring and analysis.

Research Design and Methods: The proposed work will occur in three stages. In the first two stages (pilot testing of task modifications and computerization in Stage 1; evaluating the test-retest reliability in Stage 2), children will complete the newly developed computerized assessment of executive functioning (EF). In the third stage of work, children will complete the newly developed EF computerized assessment at the first visit and will complete one of three possible subsets of validation tasks (EF, intelligence/processing speed, or academic achievement) at the second visit using a planned missing design. This design facilitates a broader set of validation activities while minimizing testing burden for individual children.

Control Condition: Due to the nature of the research design, there is no control condition.

Key Measures: The Stage 3 validation tasks will include direct assessments of children's executive function, intelligence, processing speed, and academic achievement using existing standardized measures of these skills. Similarly, teachers will complete one of three possible sets of rating scales for each skill.

Data Analytic Strategy: A combination of item response theory and structural equation modeling will be used to evaluate the psychometric properties of the computerized battery and evaluate the construct and criterion validity of the battery.

Products and Publications

Journal article, monograph, or newsletter

Willoughby, M.T. (2014). Formative Versus Reflective Measurement of Executive Function Tasks: Response to Commentaries and Another Perspective. Measurement: Interdisciplinary Research and Perspectives, 12(4): 173–178. doi:10.1080/15366367.2014.981074 Full text

Willoughby, M.T. and Blair, C.B. (2015). Measuring Executive Function in Early Childhood: A Case for Formative Measurement. Psychological Assessment, 28(3): 319–330. doi:10.1037/pas0000152 Full text

Willoughby, M.T., Holochwost, S., Blanton, Z., and Blair, C.B. (2014). Executive Functions: Formative versus Reflective Measurement. Measurement: Interdisciplinary Research and Perspectives, 12(3): 69–95. doi:10.1080/15366367.2014.929453