Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Project SCREEN: Validation of a Free-Access Screening Tool for K-12 Educators to Screen Students for Internalizing and Externalizing Behavior Patterns
Center: NCSER Year: 2019
Principal Investigator: Lane, Kathleen Awardee: University of Kansas
Program: Social, Emotional, and Behavioral Competence      [Program Details]
Award Period: 3 years (07/01/2019-06/30/2022) Award Amount: $1,399,959
Type: Measurement Award Number: R324A190013

Co-Principal Investigator: Oakes, Wendy

Purpose: The purpose of this project is to validate the Student Risk Screening Scale for Internalizing and Externalizing Behaviors (SRSS-IE) for identifying K-12 students who are at risk for internalizing and/or externalizing behavior patterns. To prevent students with and at risk for emotional and behavioral disorders (EBD) from developing long-term negative consequences such as school failure and dropout, it is essential that students be identified early to facilitate the identification, implementation, and evaluation of proper interventions. Although screening tools exist, schools need more options for low-cost, feasible screening tools to identify externalizing and internalizing behavior problems in schools. The research team has developed the initial version of the SRSS-IE, a free-access screening tool, but additional research is needed to evaluate its psychometric properties.

Project Activities: The research team will conduct a series of studies that involves collecting new data in the first year of the grant and analyzing these and existing data from SRSS-IE studies conducted to date. Five sets of analyses will be conducted including (1) measurement invariance to ensure that the same internalizing and externalizing constructs are being measured across specified groups (gender, race/ethnicity, disability status, and grade level); (2) internal consistency to determine how well the SRSS-IE is measuring targeted behaviors across items, subscales, and student ability levels; (3) classification accuracy to compare the accuracy of SRSS-IE screening to results obtained using other established and well-researched screening tools; (4) convergent validity of the SRSS-IE with other validated screening measures and predictive validity of the tool with other year-end indicators of academic, behavioral, and social outcomes; and (5) existence of any bias with respect to gender, ethnicity/race, and disability status.

Products: The products of this project will include a fully developed and validated SRSS-IE as a screening tool to identify K-12 students who are at risk for internalizing and/or externalizing behavior patterns. The project will result in peer-reviewed presentations and publications as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.

Structured Abstract

Setting: Data were, and will be, collected from elementary, middle, and high schools across the U.S., including California, Arizona, Kansas, Missouri, Tennessee, North Carolina, Pennsylvania, and Vermont.

Sample: Extant data includes ratings of 36,600 elementary, middle, and high school students collected from 1,268 teachers. An additional 19,000 teacher ratings of students will be collected in Year 1. Participating elementary school teachers rate all students enrolled for at least 30 days in their homeroom classes and middle and high school teachers rate students in one class period.

Assessment: The SRSS-IE is a free-access, teacher-completed, 12-item scale for detecting K-12 students with or at risk for externalizing and/or internalizing behaviors. All items are rated on the same 4-point Likert-type scale: never = 0, occasionally = 1, sometimes = 2, frequently = 3. Externalizing behavior items include (1) steal; (2) lie, cheat, sneak; (3) behavior problem; (4) peer rejection; (5) low academic achievement; (6) negative attitude; and (7) aggressive behavior. Internalizing behavior items include (8) emotionally flat; (9) shy, withdrawn; (10) sad, depressed; (11) anxious; and (12) lonely. Items are summed for each student and the student's total score falls into one of three categories of risk: low (0-3), moderate (4-8), or high (9-21).

Research Design and Methods: In the first year of the grant, participating districts will conduct the SRSS-IE in the fall, winter, and spring to collect student ratings as part of regular school practices. The research team will then randomly select four students per classroom to recruit for additional teacher screening with another established behavioral screening tool. A series of analyses will be conducted, separately by grade level span (K-5, 6-8, 9-12), using data from existing SRSS-IE studies, as well the newly collected data. First, the research team will conduct a series of analyses to assess measurement invariance across ethnicity/race, gender, disability status, and grade level. They will also examine reliability across time and rater as well as internal consistency using classic test theory and item response theory. The team will then conduct classification accuracy analyses to contrast students with low- versus high-risk scores according to SRSS-IE subscales and to examine these scores with scores obtained by other established, well-researched screening tools. They will conduct convergent validity analyses to compare the SRSS-IE with other validated screening measures and predictive validity analyses to examine the ability of the SRSS-IE to predict year-end student academic and social-behavioral outcomes. The researchers will also examine the extent to which SRSS-IE scores predict referrals to and placement in special education. Finally, the SRSS-IE will be examined for differential item functioning across gender, ethnicity/race, and disability status to ensure the screening tool is free from bias.

Control Condition: Due to the nature of the research design, there is no control condition.

Key Measures: The Teacher's Report Form will be used as an independent criterion tool of externalizing and internalizing behaviors for determining concurrent validity and classification accuracy. Teachers will complete a demographic form to indicate gender, years of experience, highest degree obtained, and training in classroom management. School records will be collected on student outcomes including office discipline referrals, suspensions, nurse visits, reading scores, course failures, and GPA.

Data Analytic Strategy: A series of multilevel IRT analyses that take into account the nested structure of the data will be used to examine measurement invariance. To assess internal consistency, item fit functions (IFF), item information functions (IIF), test information functions (TIF), and test standard error (TSE) plots will be created to examine the degree to which individual items and subscales provide information across all student ability levels. The team will compute both Cronbach alpha and omega total to assess reliability. Validity will be examined by computing a series of random effects negative binomial regressions. All reliability and validity analyses will be repeated for demographic subgroups by ethnicity/race, gender, and disability status within each grade level span. IRT latent scale scores and ROC curve analyses will be used to determine classification accuracy. Differential item functioning will be tested by using the likelihood ratio test.

Related IES Projects: Enhancing Ci3T: Building Professional Capacity for High Fidelity Implementation to Support Students' Educational Outcomes (Project ENHANCE) (R324N190002)