Project Activities
The research team will conduct a series of studies that involves collecting new data in the first year of the grant and analyzing these and existing data from SRSS-IE studies conducted to date. Five sets of analyses will be conducted including (1) measurement invariance to ensure that the same internalizing and externalizing constructs are being measured across specified groups (gender, race/ethnicity, disability status, and grade level); (2) internal consistency to determine how well the SRSS-IE is measuring targeted behaviors across items, subscales, and student ability levels; (3) classification accuracy to compare the accuracy of SRSS-IE screening to results obtained using other established and well-researched screening tools; (4) convergent validity of the SRSS-IE with other validated screening measures and predictive validity of the tool with other year-end indicators of academic, behavioral, and social outcomes; and (5) existence of any bias with respect to gender, ethnicity/race, and disability status.
Structured Abstract
Setting
Data were, and will be, collected from elementary, middle, and high schools across the U.S., including California, Arizona, Kansas, Missouri, Tennessee, North Carolina, Pennsylvania, and Vermont.
Sample
Extant data includes ratings of 36,600 elementary, middle, and high school students collected from 1,268 teachers. An additional 19,000 teacher ratings of students will be collected in Year 1. Participating elementary school teachers rate all students enrolled for at least 30 days in their homeroom classes and middle and high school teachers rate students in one class period.
Assessment
The SRSS-IE is a free-access, teacher-completed, 12-item scale for detecting K-12 students with or at risk for externalizing and/or internalizing behaviors. All items are rated on the same 4-point Likert-type scale: never = 0, occasionally = 1, sometimes = 2, frequently = 3. Externalizing behavior items include (1) steal; (2) lie, cheat, sneak; (3) behavior problem; (4) peer rejection; (5) low academic achievement; (6) negative attitude; and (7) aggressive behavior. Internalizing behavior items include (8) emotionally flat; (9) shy, withdrawn; (10) sad, depressed; (11) anxious; and (12) lonely. Items are summed for each student and the student's total score falls into one of three categories of risk: low (0-3), moderate (4-8), or high (9-21).
Research design and methods
In the first year of the grant, participating districts will conduct the SRSS-IE in the fall, winter, and spring to collect student ratings as part of regular school practices. The research team will then randomly select four students per classroom to recruit for additional teacher screening with another established behavioral screening tool. A series of analyses will be conducted, separately by grade level span (K-5, 6-8, 9-12), using data from existing SRSS-IE studies, as well the newly collected data. First, the research team will conduct a series of analyses to assess measurement invariance across ethnicity/race, gender, disability status, and grade level. They will also examine reliability across time and rater as well as internal consistency using classic test theory and item response theory. The team will then conduct classification accuracy analyses to contrast students with low- versus high-risk scores according to SRSS-IE subscales and to examine these scores with scores obtained by other established, well-researched screening tools. They will conduct convergent validity analyses to compare the SRSS-IE with other validated screening measures and predictive validity analyses to examine the ability of the SRSS-IE to predict year-end student academic and social-behavioral outcomes. The researchers will also examine the extent to which SRSS-IE scores predict referrals to and placement in special education. Finally, the SRSS-IE will be examined for differential item functioning across gender, ethnicity/race, and disability status to ensure the screening tool is free from bias.
Control condition
Due to the nature of the research design, there is no control condition.
Key measures
The Teacher's Report Form will be used as an independent criterion tool of externalizing and internalizing behaviors for determining concurrent validity and classification accuracy. Teachers will complete a demographic form to indicate gender, years of experience, highest degree obtained, and training in classroom management. School records will be collected on student outcomes including office discipline referrals, suspensions, nurse visits, reading scores, course failures, and GPA.
Data analytic strategy
A series of multilevel IRT analyses that take into account the nested structure of the data will be used to examine measurement invariance. To assess internal consistency, item fit functions (IFF), item information functions (IIF), test information functions (TIF), and test standard error (TSE) plots will be created to examine the degree to which individual items and subscales provide information across all student ability levels. The team will compute both Cronbach alpha and omega total to assess reliability. Validity will be examined by computing a series of random effects negative binomial regressions. All reliability and validity analyses will be repeated for demographic subgroups by ethnicity/race, gender, and disability status within each grade level span. IRT latent scale scores and ROC curve analyses will be used to determine classification accuracy. Differential item functioning will be tested by using the likelihood ratio test.
People and institutions involved
IES program contact(s)
Products and publications
Products: The products of this project will include a fully developed and validated SRSS-IE as a screening tool to identify K-12 students who are at risk for internalizing and/or externalizing behavior patterns. The project will result in peer-reviewed presentations and publications as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.
Related projects
Supplemental information
Co-Principal Investigator: Oakes, Wendy Peia
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.