Previous Award Number: R324B060033
Previous Awardee: University of Georgia

Purpose: The purpose of this project is to develop and validate a five-minute screening assessment to be used by teachers to identify elementary and middle school children with behavioral or emotional problems that predispose them to academic failure. The study will build upon pilot research to develop a screener from the longer form of the Behavioral Assessment System for Children, second edition (BASC-2; Reynolds & Kamphaus, 2004), currently used widely in the U.S. and abroad.

Setting: This project will be implemented in large, linguistically, socio-economically, and ethnically diverse school districts in California and Maryland.

Population: Students between the ages of 5 and 13 in the kindergarten through seventh grades are targeted for this study. It is expected that teacher ratings and parent questionnaire information will be obtained from approximately 1400 students in each of three cohorts for a total of 3,600 students over three years. Two boys and two girls will be selected at random for participation in each class at each grade level within each school. Within each school, information will be gathered from teachers as well as collected from archival data on students (e.g., grades, number of absences, standardized test scores).

Structured Abstract

Project Activities: The research investigation will be conducted sequentially, in three phases: (1) develop the screener; (2) assess the psychometric quality of the screener, resulting in multiple measures of reliability and validity; (3) estimate the screener's long-term predictive validity using an accelerated longitudinal design. The outcome is a screening instrument that will take the teacher five minutes or less per child, and have adequate reliability and predictive validity as judged by standards set by measurement science research precedents. Statistical methods such as principal components analysis, reliability and item analysis, Receiver Operating Characteristic Curve analysis, and multiple regression will be employed to measure the reliability and validity, and estimate the predictive validity of the developed screener.

Products: The expected outcomes from this study include: (a) A developed and validated teacher-rated screener to accurately identify children with behavioral or emotional problems; (b) published reports on the results from the validity studies and the psychometric quality of the screener for identifying children with behavioral and/or emotional problems; (c) published reports on the longer-term predictive validity of the developed screener based on the analysis of the longitudinal investigation; and (d) Screener/Assessment documentation and forms to be used for special education classification.

Research Design and Methods: The research investigation will be conducted sequentially, in three phases: (1) develop the screener; (2) assess the psychometric quality of the screener, resulting in multiple measures of reliability and validity; (3) estimate the screener's long-term predictive validity using an accelerated longitudinal design. The outcome is a screening instrument that will take the teacher five minutes or less per child, and have adequate reliability and predictive validity as judged by standards set by measurement science research precedents. Statistical methods such as principal components analysis, reliability and item analysis, Receiver Operating Characteristic Curve analysis, and multiple regression will be employed to measure the reliability, validity, and estimate the predictive validity of the developed screener.

Key Measures: The screener is based on widely used comprehensive measure of children's behavioral and emotional adjustment, the Behavioral Assessment System for Children-2 (BASC), of which the PI is a co-developer. The BASC-2 consists of the Self-Report of Personality (SRP), Parent Rating Scales (PRS), and Teacher Rating Scales (TRS), a Student Observation System (BASC SOS), and a structured developmental history (BASC SDH). The items on the TRS are rated on a four-point response scale of frequency ranging from "Never" to "Almost Always." The full TRS forms can be completed in about 20 minutes, and scales can cover the age range of 2 ½ through 18. The proposed investigation will include larger samples than included on previous work with the BASC, create a screener that spans ages 5-13, draws items from a modern, broader, and higher quality item pool, assess incremental validity, and assess predictive validity over an eight year period among other improvements over the BASC.

Data Analytic Strategy: The data analysis takes place in three phases: substantive, structural and external. The first stage, the substantive stage, is concerned with how the construct(s) is defined, both theoretically and empirically. For this stage, principal components factor analysis will be used to develop the screener items and the results will be compared to factors of child adjustment derived from the literature. The structural component provides assurances that the results can be replicated if the instrument was given to the same individuals under similar conditions. Three reliability coefficients will be estimated: internal consistency, short and long term test-retest, and inter-rater. In addition, criterion and predictive validity evidence will provide support that the inferences made from the screener scores are trustworthy (e.g., usefulness of screener scores to make decisions about child behavior). The external phase will examine the relationships between screener scores and additional instruments to assess the usefulness of the screener for identifying children at risk for behavioral, emotional, and academic adjustment in schools. To measure the accuracy of discrimination between children with emotional, behavioral, or academic problems and those without, Receiver Operating Curve (ROC) analysis will be conducted. Known group validity analyses will also be performed to examine the ability of the screener to identify children with disabilities from those who are developing normally.

Intervention: Due to the nature of the research, there is no intervention.

Control Condition: Due to the nature of the research, there is no control condition.

Products and Publications

Journal article, monograph, or newsletter

Chin, J., Dowdy, E., and Quirk, M. (2013). Universal Screening in Middle School: Examining the Behavioral and Emotional Screening System. Journal of Psychoeducational Assessment, 31(1): 53–60. doi:10.1177/0734282912448137

Dever, B., Raines, T., and Dowdy, E. (2016). Factor Structure and Differential Item Functioning of the BASC-2 BESS Spanish Language Parent Form. School Psychology Quarterly, 31(2): 213–225. doi:10.1037/spq0000131

Dever, B.V., Mays, K.L., Kamphaus, R.W., and Dowdy, E. (2012). The Factor Structure of the BASC-2 Behavioral and Emotional Screening System Teacher Form, Child/Adolescent. Journal of Psychoeducational Assessment, 30(5): 488–495. doi:10.1177/0734282912438869

DiStefano, C., and Morgan, G. (2010). Evaluation of the Bess TRS-CA Using the Rasch Rating Scale Model. School Psychology Quarterly, 25(4): 202–212. doi:10.1037/a0021509

DiStefano, C., and Morgan, G. (2011). Examining Classification Criteria: A Comparison of Three Cut Score Methods. Psychological Assessment, 23(2): 354–363. doi:10.1037/a0021745

Dowdy, E., Chin, J., Twyford, J., and Dever, B. (2011). A Factor Analytic Investigation of the BASC-2 Behavioral and Emotional Screening System Parent Form: Psychometric Properties, Practical Implications, and Future Directions. Journal of School Psychology, 49(3): 265–280. doi:10.1016/j.jsp.2011.03.005

Dowdy, E., Dever, B. V., Raines, T., and Moffa, K. (2016). A Preliminary Investigation into the Added Value of Multiple Gates and Informants in Universal Screening for Behavioral and Emotional Risk. Journal of Applied School Psychology, 32(2): 178–198. doi:10.1080/15377903.2016.1165327 Full text

Dowdy, E., Dever, B., DiStefano, C., and Chin, J. (2011). Screening for Emotional and Behavioral Risk Among Students With Limited English Proficiency. School Psychology Quarterly, 26(1): 14–26. doi:10.1037/a0022072

Dowdy, E., Doane, K., Eklund, K., and Dever, B. (2013). A Comparison of Teacher Nomination and Screening to Identify Behavioral and Emotional Risk Within a Sample of Underrepresented Students. Journal of Emotional and Behavioral Disorders, 21(2): 127–137. doi:10.1177/1063426611417627

Dowdy, E., Kamphaus, R.W., Abdou, A.S., and Twyford, J. (2013). Detection of Symptoms of Prevalent Mental Health Disorders of Childhood With the Parent Form of the Behavioral and Emotional Screening System. Assessment for Effective Intervention, 38(3): 192–198. doi:10.1177/1534508412447009

Dowdy, E., Ritchey, K., and Kamphaus, R.W. (2010). School-Based Screening: A Population-Based Approach to Inform and Monitor Children's Mental Health Needs. School Mental Health, 2(4): 166–176. doi:10.1007/s12310–010–9036–3

Dowdy, E., Twyford, J., Chin, J., Kamphaus, R., and Mays, K. (2011). Factor Structure of the BASC-2 Behavioral and Emotional Screening System Student Form. Psychological Assessment, 23(2): 379–387. doi:10.1037/a0021843

Eklund, K., and Dowdy, E. (2014). Screening for Behavioral and Emotional Risk Versus Traditional School Identification Methods. School Mental Health, 6(1): 40–49. doi:10.1007/s12310–013–9109–1

Kamphaus, R.W., DiStefano, C., Dowdy, E., Eklund, K., and Dunn, A. (2010). Determining the Presence of a Problem: Comparing Two Approaches for Detecting Youth Behavioral Risk. School Psychology Review, 39(3): 395–407.