Co-Principal Investigators: French, Brian; Gotch, Chad; Poppen, Marcus

Purpose: The purpose of this measurement project is to further refine and validate a low-cost, efficient, and proactive assessment to address student truancy and dropout. The Washington Assessment of the Risks and Needs of Students(WARNS) is used in middle and high schools to identify factors that contribute to truancy and are malleable in response to intervention. The WARNS provide a framework for school personnel to generate conversations and personalized action plans with youth for improving attendance and school outcomes based on their personalized risks and needs. The WARNS has strong psychometric properties and is used throughout Washington state. With this refinement and development project, the research team intends to improve its validity and user-friendliness such that it is attractive and affordable to school districts nationwide.

Project Activities: The research team will expand the domain coverage of WARNS by adding strengths-based item content. They will also refine and norm the middle school version, improve assessment accessibility features, develop innovative and holistic modes for communicating results, and generate further support for use of the instrument for rapport-building and action planning with truant and at-risk students.

Products: The primary product in this project will be a ready-to-implement, concise, web-based instrument that engages both middle and high school students at risk of school failure due to truancy and related factors. The instrument will generate normative scores on six risk/needs domains and a cumulative total risk factor to assess students' risk for school failure, engage in solution-focused conversations, and generate action plans relevant to issues that cause absenteeism. Score reports will display immediately upon completion of the 40-item instrument to portray the student in terms of level of risk (low, medium, and high). Narrative reports will foster student self-reflection that can facilitate participation in the social process of identifying and overcoming barriers to school attendance and success.

Structured Abstract

Setting: The study will take place in public school districts in Washington state that use the WARNS to improve outcomes for students at risk for school failure due to excessive unexcused absences and other risk factors.

Sample: Participants include students in Washington State who complete the WARNS in routine use during the span of the project. The sample will include approximately 7,000 high school students and 3,000 middle school students representative of a student population at risk for school failure due to absenteeism and associated risk factors. Sampling procedures for analyses will ensure that the sample is representative in terms of race, ethnicity, and socioeconomic background. Analyses concerned with predictive validity will use a subsample of 400 students recruited from partnering districts, for which academic records will be obtained and linked to WARNS scores.

Assessment: The WARNS is a 40-item, web-based instrument that promotes positive outcomes for adolescents through assessment of functioning in social and behavioral domains associated with school truancy and dropout. Derived from life-course criminology theory, the WARNS provides measures of a student's aggression-defiance, depression-anxiety, substance abuse, peer deviance, school engagement, and family environment. In addition to these scores, the assessment generates a total risk score.

Research Design and Methods: The research team will use an iterative design and refinement process based on contemporary test development and psychometric methods to strengthen the instrument and the documentation of its functioning in applied school settings. They will also investigate score use through both quantitative and qualitative analyses.

Control Condition: Due to the nature of this study, there is no control condition.

Key Measures: Key outcome measures reflect five validation inferences: domain description (adequate construct coverage of salient risk and need factors), scoring (accurate representation of differential endorsement of the items relevant to the domain), generalization (representation of the relevant assessment universe), extrapolation (indication of an accurate level of the student on the domain measured that is consistent with how that risk/need manifests in their lives), and implications (association with positive student outcomes). Student outcome variables will include year-end grade point average, student expulsions, school suspensions, unexcused absences, grade retention, and state test scores.

Data Analytic Strategy: The research team will engage in development, revision, and analytic work iteratively, first focusing on the high school version and cost analysis ingredients, and then replicating procedures for the middle school version. They will examine item difficulty, item discrimination, measurement invariance, internal consistency reliability along with innovative and holistic approaches to communication of scores. They will obtain qualitative input concerning the validation process from a group of measurement experts who will comprise a technical advisory committee. They will get qualitative input concerning innovative practices for score reporting from a stakeholder panel of school counselors.

Cost Analysis: The research team will use the "ingredients approach" to assess costs of WARNS use to school districts.